Items disappearing

Jerud

In the last few days I’ve noticed some missing – or possibly mis-synced – items.
While it’s possible this is due to my own careless clicks or keystrokes, it’s happened twice now in as many days – compared to zero times in the preceding 6 years of using Legend, so I’m thinking this isn’t me.

The first time was yesterday: I had a 2nd-level branch that was named using (only) a OneNote link (this is an “attachment”). I renamed the section in Onenote, copied the new link, pasted it at the end of the existing Branch name, Then backspace-d to delete the old link. If you’ve never tried operations like this around ‘attachments’, then I’ll just point out: it can be tricky. The ‘attachment’ behaves sort of like text but the enclosing markdown it is also sorta-involved, so you can end up with unexpected cursor movements, deleting more than you meant, etc. But this isn’t my first rodeo and everything looked fine. Later that evening I returned to that section of my outline to find the entire branch I had previously renamed was now completely gone. Now: maybe I pressed ‘delete’ just one time too many and blew away the whole branch – but where was my “delete multiple items” Toast if that was the case? I dismissed this as my mistake and restored the branch from backup, renamed it again (same way), and had no further trouble.
Now today: I went to a small Project I know I had fleshed out last week, with several Tasks and a number of subtasks (numlisted). When I got to it, I found just two Tasks, both being old versions I had sketched in as placeholders weeks ago. None of the numlist items were to be found. I did several global searches on keywords I remembered, and of course opened it in a popup with no filters – didn’t find anything.

Both these examples happened in outline regions I’ve been working in recently, so I actually noticed them. Neither of them are crushing losses. But it’s scary to consider where else this may have happened – and I’ll simply never know – that is, not until I miss a payment or appointment, or omit an important detail I was counting on having logged.

I can’t help but wonder if this is related to mobile: I saw the recent mobile update, so in a fit of hopefulness I reinstalled the mobile app on my ipad and iphone a few days ago. Things seemed to be working okay; I was able to check my tasklist while out-and-about (I had missed that). But after this, I’ll be deleting those apps again.

@Jay, I’m not sure what useful details I can provide you, but if you DM me I can point you to the problematic items in my outline.

Windows 11 desktop app, stable.
iOS 17.0.2 - iPhone 13 mini
PadOS 17.0.1 - iPad 7th gen

Jay

I think the latest update should fix these problems. There was a window of time in which it would start a save while in the middle of another save and that could cause some strange behavior. So now that is fixed and after hours of bashing against it and trying everything I can think of, I can’t find any more sync problems anywhere.

I don’t think the mobile apps would be the problem. But that bug was exacerbated by multiple instances being online at the same time. I’ll test the latest version with mobile apps open at the same time just in case.

Jerud

OK, I’ll see if the issue reproduces, and will check using mobile as well.

It’s pretty common for me to open mobile while I still have desktop open, which maybe creates the “multiple instances” situation. I do that b/c I need to be sure the mobile app updates on my home wifi; I often don’t have reliable signal while away.

I just restarted from a full system shutdown and launched Legend; no app update appeared. I did a force-reload and then a manual check for updates – nothing. Any reason the last update isn’t pushing to me? I am on stable not beta.

Jay

It was only in beta but I just released it to stable.

It should absolutely be fine to have many devices connected at the same time, and I think it is now! I’ve been testing everything I can think of for a couple days now and can’t find any more issues.

Jerud

Marking resolved; I don’t see any obvious repeat problems. Will reopen if necessary.

Jerud

Okay definitely NOT SOLVED.

I put Legend back on my phone and synced up with a bunch of changes I had made on desktop last night. I went into town and added some notes to items as I was completing them. But while I was doing that, I noticed a project I had made last night simply wasn’t appearing. When I got home and could pay better attention I went hunting for it in the mobile app, turning off filters etc. to make sure it wasn’t hidden. Nope, not there.

I booted up my laptop and turned off wifi before logging in to prevent sync, then went to check for the missing project. Yep: it was just where I had expected it to be. So I hesitantly turned wifi back on and watched Legend sync, whereupon I watched the following things happen:

The items I had added in mobile synced back over to desktop: expected
The project that had never synced to mobile from desktop now disappeared from desktop
On the same Board, I had a Pane viewing one of my top-level branches which contained four sub-branches. Three of them just plain disappeared!!! I never even opened those branches in mobile…why?!?!
Elsewhere in that document, yet again I had a smattering of zombie items reappear
- These are items that I had just yesterday moved out of my “inbox” area at the top of the outline and filed into the body of the outline … now they’re back, but at the bottom of the outline instead of the top, where they had been.
- I saw them first when viewing my inbox in mobile, then after I synced the desktop app, they all turned into broken links – the item content is replaced with just the item ID (gibberish like: -MgS23KZdOCAenS2T_1u), which is displayed in [[link]] formatting – even though these had just been plain items, not links or mirrors.
- I’m certain that some of the zombies had been organized into one of the branches that got blown away. I’m not sure if other zombies were put into the other two vaporized branches, so it’s not clear if the mechanism that spawns zombies is the same one that blows away outline branches, of if that’s just a coincidence.
The rest of that document appears OK but here again…I wouldn’t know how many things have been quietly blown away at this point. The scope of the mis-syncing doesn’t appear to be limited to where I actually worked, which means defects could be anywhere.
The web app appears to have the same sync state as on desktop. I didn’t check it before using the mobile app so I can’t be positive that my desktop changes from last night actually synced up to the server, but I have no reason to think they didn’t, either.

This is all in STABLE.

This is the biggest and most recent step backwards from what used to be a totally solid sync experience in the app. I left beta because I couldn’t afford all the time I was losing to fixes in Legend, and now here I am in stable with the same problems. Where am I supposed to go from here?

@Jay, is this really only happening to a handful of us? Is it connected to older accounts, or documents over a certain size? I can’t get through two days without having to fix my outlines – unless I sandbox Legend to strictly local operation. Is there some feature that’s been added in the past year or two that is making sync so sketchy? Maybe mirrors? Because at this point I’d say you’re better off killing whatever “feature” is breaking your sync system, even if it means cutting back functionality – just to get the app reliable.

A productivity app without reliable sync isn’t worth dirt.

Jay

I think the cause of these issues should be fixed in the latest version. There were some edge cases around moving items between documents while offline that I believe were the cause.

And I added a new “Moved” pane view type which I believe should show some of the items that have gone missing, and you can restore them from there (like the Archived/Deleted views). So hopefully that should get all your missing items back. It’s not technically possible to delete data from the database, only to mark it as archived/deleted/moved. So I think the missing items were marked as moved and should be in that view.

Unfortunately yes, the sync errors are getting increasingly edge case, but it’s the edges that I think you in particular are more likely to hit, like being offline or on bad connections. And moving items between documents has always been fragile, and I think that was the cause of your issues based on the errors I saw in the error reporter? I am focusing my testing now on going between offline and online. This update is a big improvement and I can’t find any other problems, but I will keep looking until we’re at 100% perfect sync.

The update is just in beta right now, but I’ll test it some more tomorrow and release it to stable.

Jerud

I guess I didn’t think of myself as edge-case, but knowing that does make it easier to understand why these bugs seem more prevalent to me than they really are.

It does feel a lot like every time I use the mobile apps something bad happens to my Documents. If I just skip the mobile app I have to rely on other apps, which leads to using Legend less. But if I don’t try to use the mobile app, whatever is causing this havoc will never be found and fixed. That’s not an attractive pair of options : (

I got the “moved” Pane and tried it out – no luck. I do see remnants of a previous organizational approach in there, which were later moved out – so the Pane seems to work. But none of the recently-missing content was there. The three branches were “permanent” headings in my outline, so I know exactly what they were each named. No hits came up when searching on those names or when just scrolling through.

I am trying to restore to my last local state and will see how that goes.

I guess I never really grasped what “items can’t be deleted” means until I saw these items in the Moved Pane. Does this mean that every Document will continuously grow, even if items are deleted or moved out of it – because those items are never actually removed once they are created – just marked as deleted or moved?

Jay

The documents grow in the backend but they’re not loaded at all until you open a “Moved” or “Deleted” pane. Once we get sync fully 100% stable I have a project planned to optimize that further.

Jerud

Jay The documents grow in the backend but they’re not loaded at all until you open a “Moved” or “Deleted” pane. Once we get sync fully 100% stable I have a project planned to optimize that further.

Cool, that would be nice. I’d love to know I can fully 100% eradicate the vestiges of old “systems” that are lying around within my Documents. And, ideally, the old Documents in the trash, too.

wecanseeformiles

I’ve been getting a client up to speed on Legend. We had a critical connection error on his Macbook M1 today during an update which didn’t go away. Re-installing the app didn’t solve the problem but deleting the app with AppCleaner Pro and then re-installing got it working today. We had Legend working on another of his computers at the same time.

However I noticed just being leaving that one or two of his top-level projects had disappeared. I will investigate further tomorrow but wondered if this was a similar issue.

Jay

wecanseeformiles What was the critical connection error? That was a problem in Legend that prevented it from opening?

wecanseeformiles

@jay I found that the items that had disappeared were in your ‘Delete’ folder so I manually restored them. We might have accidentally deleted them but they were items from different projects so that seems unlikely to me.

Unfortunately I didn’t take a screen-grab of the load error message but it announced a ‘Critical Connection error…’ after which the Legend remained blank except for this error message.

This happened with a client’s Legend, not my own one. He does have intermittent internetconnection issues so that might be something.