Document syncing between platforms inconsistencies

SamMarrocco

Documents, their order, visibility and existence are very buggy and inconsistent between OS and multiple instances of the Legend app. I’ve been able to reproduce all of these issues many times:

I performed the following tests, keeping all machines/platforms/instance of Legend visible at the same time to allow me to watch what was happening in real time. This was the setup:

PC #1 (Windows 11 running newest Legend standalone app)
PC #2 (Windows 11 running newest Legend standalone app)
Macbook Pro (2018 Intel running OSX Monterey 12.6, Legend Standalone app)
iPad Pro running newest iOS and Legend App
iPhone 14 Pro Max running newest iOS and Legend App

From the start, I have several documents on my iPad and iPhone that I am unable to delete (because of the lack of that functionality in Legend on those platforms. So I’m stuck with those Documents. For discussion sake, I’ll ignore them in this report.

Beginning State: Both PCs, the Mac, iPad and iPhone all have a single document I created while all apps were running. The document was named DEFAULT and created on PC#1. All instances of Legend show that document.

Test #1: Make a new document on Windows PC #1 Named DEFAULT
RESULT: DEFAULT doc appears on all platforms (PC 1,2, MacBook, iPhone and iPad)

Test #2
Quit all Legends on all platforms except for PC #1.
Create a new document named FRED on PC#1 Legend.
FRED document appears on PC#1 Legend at the TOP of the documents list.
Launch all the other platform Legends. FRED document is NOT at the top of the document list.
BUG: New documents appear to appear at the top of the document list ONLY on the instance/platform of Legend on which they were created. If there are other documents, it will appear anywhere in the documents list, but usually at the bottom of the list.

Test #3
On PC#1, change the order of the DEFAULT and FRED documents.
BUG: The order of the document on the other platforms does NOT change to match.

Test #4
Quit all the Legends on all platforms except for PC #1
Delete the FRED document on PC#1 (Leaving only the DEFAULT Document on PC#1)
Launch all the other platform Legends.
BUG: Macbook pro, iPhone, PC#2 and iPad all still show Fred Document along with Default Document. The only machine showing that the deletion of Fred tool place is PC#1 (the machine on which it was deleted).

Test #5
Quit all the Legends on all platforms except for MacBook Pro
Delete the FRED document on Macbook Pro (Leaving only the DEFAULT Document on PC#1)
Launch all the other platform Legends.
BUG: PC#2 show that FRED document has been deleted along with MacBook Pro. However, iPad, iPhone and PC#2 still show a FRED document along with the DEFAULT document.

This is just a few of the issues I’ve found. The inability to delete a shared document by another Legend instance COMBINED with the inability to delete a document on iPhone and iPad (iOS) apps locally within the app means that you can never delete a document on iOS platforms UNLESS they are running while the Document is deleted remotely by another platform. The Trash ‘bins’ for documents are out of sync between platforms as well. Theorectically, if all I ever create are shared documents, and they are deleted, the trash doc bins on all platforms should match in content (they do not) and contain the same number of trashed documents (they do not). They are all drastically different in content and count.

Is there a way to ‘delete everything’ in a Legend account, clearing out every document you have? It’s almost impossible to troubleshoot at this point since all my iOS apps have dozens of docs I am unable to delete…?

LauraH

I don’t have it nearly as well documented as Sam did. (THANK YOU FOR the detailed documentation SAM!!)

But I’m seeing similar weird behavior and problems syncing between platforms.

Windows machine showing items as deleted (which shouldn’t be!)
Web version shows the items correctly
iphone also seems to be correct

Logging out and back in again on the windows version isn’t fixing it. I’ve checked for updates on all versions - all are up to date…

SamMarrocco

You’re welcome Laura!

After some dialogs with Jay, I wanted to include a summary and the ‘workaround’ here to complete the thread in case others are experiencing the same issues. Note that these are specific to ‘cloud/shared’ documents added for viewing on all platform/OSes.

-If an iOS, Windows, or OSX Legend Native App is on a device, but it not actively running it will not ‘notice’ Document Deletions and changes.

-Adding a Document to a particular device adds it to the Top of the Document List on that device. It is added to the Bottom of the Document List on all other devices (assuming you can see the new document on the other devices).

-Deleting a Document behaves in a similar fashion, only showing expected behavior on the Device on which you perform the operation. If a non-running app on another device is started after a Document was deleted somewhere else, the document will still be present on the non-running app when it is started. On iOS, this leaves you with a apparently shared document that you cannot delete, because the iOS version of Legend does not have a function for deleting documents.

-The Trash ‘bins’ are subject to this same type of inconsistency and I haven’t attempted to discern that.

Jay suggested that I Log-Out of my Legend apps on the platforms that were out of sync. I did that, and after signing in again they synchronized correctly with the cloud data being show on my other OS platforms. Obviously, this is undesirable to have to do every time you create something on one platform and the others aren’t running, but it does give a ‘fix’ to get you over the hump of having many unsynchronized documents across all your platforms. He also suggested that a new synchronization method was coming in the future that (should) fix the issues.

As I’m still evaluating Legend, I have not attempted to determine if this type of synchronization behavior is also demonstrated in the contents of documents, or if it is limited to the existence of the documents themselves.

SamMarrocco

The synchronization bugs/issues I’ve been tracking are far worse than I thought.

I thought I might be able to use Jay’s ‘workaround’ of signing out/signing in to each instance after my documents were created to cure the ’my docs aren’t showing up in all platforms/instances of Legend-bugs’. Although tedious and repetitive, it worked and gave me a synchronized Legend app on all my devices/platforms.

So I imported my workflowy outline into several Legend Documents as OPML and formatted them all as bullets, resulting in a ‘mirror’ of my workflowy setup.

I dove into panes (my primary reason for trying to move from Workflowy to Legend). Spent an afternoon playing in there, learned the idiosyncracies inherent in some of the names & descriptions, and managed to watch my documents on my four devices live stay in sync–as long as I didn’t add or delete a document. All good so far.

Then I left the office and went home, firing up a Legend on my home pc. My five documents are now there, but one is empty of items. Odd, since my other portable devices are still showing content in that document. I signed in/out of my home PC, hoping the content would appear. Nope.

So I went on one of my devices in which the content was present, signed out of it, then in, and the content vanished from the document.

I recreated this scenario a few times to confirm it.

Here is what I’ve found:
To have usable synchronization, Every platform/device on which you use Legend (every iOS device, every Windows PC, every OSx machine) must ALWAYS, ALWAYS be running Legend LIVE while you are adding, deleting, changing the order of, or editing content in documents. If any of them aren’t currently running, document and/or their content can be lost. I’ve been unable to go a single day without losing data that, if I didn’t still have it elsewhere, would have been a total loss. If one of your platform/devices is on or off, when changes are made elsewhere, and is missing some data from a failed synchronization, it can become the ‘master’ and its incorrect content will be pushed to other platforms when they are signed in/out.

I understand Legend’s synchronization engine is planned for replacement. I’m not trying to be cruel here, but I’m not sure how anyone can use this application on more than one device and have any sort of reliability for their data. Perhaps if you only used the app on a single device (one phone, or one computer) you could count on it, but it is still IMO unusable for this reason.

This is my second attempt in a year to try to move to Legend, as I am absolutely determined to get away from Workflowy and it’s incredibly slow development pace, Perhaps I’ll give legend another try after the new synchronization engine is in place and debugged.

Jay

It’s very strange that we’ve never been able to reproduce any sync problems like this in any of our testing, but @Jorge is seeing some sync issues caused by boards failing to sync their collapsed state because his documents are extremely large, and that ends up causing cascading sync failures. Do you similarly have large documents?

We’re starting the upgrades to our sync system with boards first, which should fix that. We have that finished and in testing right now, so will try to release it soon. I will try to test it with your account’s data (even if all the text is encrypted it should have the same sync issues) and see if that fixes it. If not, this remains the highest priority thing to fix.

SamMarrocco

Laura H has also mentioned similiar issues syncing. It has been pretty easy to duplicate the issue once discovering the pattern of steps. I was very precise and calculating in my testing before posting my steps for duplication of the issue. I suspect that many people are unaware of the issue because they weren’t using it day-to-day between all those platforms, OSes, and apps while watching for it to happen, and are not creating/deleting lists as often as I was. At that point statistics may be ‘polluting’ the number of people noticing (not experiencing) the problem. I was very determined to get to the root of the sync problem before committing to using Legend,

Per your question….I’ve never noticed it being related to collapsed or expanded item states and my document size did not seem to have anything to do with the issue. For the record, most of my documents were only several items in length when noticing the issue. There were only a few documents that were a few hundred lines in length, and I eliminated those to narrow the testing parameters.

I look forward to re-evaluating Legend once the new sync engine has been tested.

Jay

Ok, thanks. I’ll try following your steps some more and see if I can find a way to reproduce it. A few hundred items wouldn’t cause the boards issue - it would have to be in the hundreds of thousands. So it must be something else.

Jerud

I haven’t been around for a while – just so dang busy – but wanted to chime in here.
Legend’s sync was always rock solid and reliable in the past but it has indeed slipped. I’m happy to see Jay is working on it.
My workflows don’t result in new documents very often so I haven’t seen the specific issue mentioned above – but as @LauraH mentions, I too have seen item-sync problems – specifically between mobile and desktop. Items added on mobile sometimes fail to appear in desktop, and some deleted items will reappear after making structure changes (moving deleting items) in mobile, once I return to desktop. I rarely even use the mobile app anymore in large part because it always creates a mess when I do. Plus other reasons that don’t belong in this thread.

I’ve mentioned an “empty trash” to Jay before and he has his reasons for not believing it is necessary…but for troubleshooting (as @SamMarrocco mentioned above), clutter management, and just plain OCD – I still really wish I could “empty trash”. I hate having documents from when I was initially playing around with Legend 4 years ago still staring at me. I do things completely differently today compared to then and there’s zero value in seeing these old docs constantly, especially since many of the names remain similar to what I use today.

Jorge

Jerud It’s not OCD… it’s’just practical reasons…. I need to cleanup sh**t that I no longer want to ever have and the best option is just to move it to a document and delete it, but it will remain there forever.

From a corporate point of view you don’t want that kind of unamanged traceability. There should be a way to clear the wastebasket completely.

But I believe this is a topic for a separate thread.

SamMarrocco

Jerud mentioning of the sync issue brings my ‘statistics’ point to the forefront: I think more people are experiencing it, but aren’t bringing it up for various reasons (i.e, I’ll deal with it later or work around it for now). I’m a big probability and statistics person, and in my facility we sometimes have users of apps say something like “but if it was that big a bug, everyone would have reported it by now”. We’ve discussed many times how this happens as:

If a piece of software has 10,000 functions.
A beta tester thinks he tested them all, but usually only tries 1000.
Each user only uses 100 out of those 10,000 functions, and every user uses a different set of 100.
Everyone complains about the buggy functions, but very few actually take the time to document it and pass it on.
By the time you do the math, it’s a miracle that anything non-major ever gets reported…..until you can’t live without that function.

I realize that this description is a bit of an understatement regarding the complexity of some software, but its point is that it’s easy for something so critical to make it to the end product and barely be noticed. Especially in these days of ’rapid, agile, scrum and other ‘trendy’ development models (I’m not implying that Legend uses those, merely that they are a contributor to major bugs falling through the cracks in the professional software market).

Most ’finished software, like computer graphics, lives in an “Uncanny Valley”.

Jerud

I’m also a believer in statistics, my background comes from the old days of “six sigma” in manufacturing. No doubt they have moved on to some other buzzwords by now but it’s the old “you can’t manage what you can’t measure” thing.

I have an extensive log of bugs ranging from “major” to “nitpicking”. I use Legend itself to track them and it’s probably my single smoothest Board / document / workflow. Copy/pasting each item to report in a new post isn’t a problem, but I’ve stopped bothering because they just drift down the list and over the event horizon. That doesn’t mean they’re not-bugs…but unless someone goes through the old posts and digs them up (obviously not going to happen) I can’t just consider them posted and scratch them off my list. They’ll never be fixed. A lot of them are daily frictions for me, so I rather want them solved – and it’s hard to imagine I’m truly the only one seeing / feeling their impact. But getting traction in the forum is tough. I’m happy to contribute my time to the success of this software that I believe in, but I’m not content to waste my time submitting and re-submitting bugs that feel like they never get seen.

So, if it’s “on fire” I will come here to report it. When I have the time, I will “batch” smaller bugs together into packages that seem logically connected (I know nothing about programming so this may not actually be helpful). The last time I did that was with mirror bugs and it was pretty effective. I’d like to do it more…but it’s pretty time consuming to compile, and If I have to put together decent repro on top of that, it’s quite a task.

Jay

I’ve been inactive here for a long time too (sorry!) but I’m back! I’ve been preparing the infrastructure for big improvements with a new state/sync system in an open source project (https://github.com/LegendApp/legend-state) and it’s now ready to start using in Legend.

I decided to scrap the large-scale rewrite I’d started last year and instead iteratively upgrade the current structure to the faster and more stable new system. I started with Board sync, which is updated in the latest beta version. And next I’ll upgrade document/item sync to use it too.

Getting sync to 100% perfect and extremely well tested is my top priority now. Once the basics are done, the new system also has support for tracking version history of individual changes, which will be better/faster than the current method of saving full backups, so I’ll plug documents into that and build a new Version History feature on it.

And once sync is 100% perfect let’s get into prioritizing the most likely statistically important bugs. I think as @SamMarrocco said, it’s likely that the most egregious bugs are never reported because they’re hit by brand new users who have no vested interest in reporting them. So I would love your help in prioritizing which issues you think are the most statistically significant.

And as @Jerud said, I felt it was pretty successful to group bugs by feature like in this thread: https://forum.legendapp.com/d/577-roadmap-discussion/6. I did that with mirrors and drag/drop and it seemed to work out really well. I think it could be good to pick a feature that needs work and just finish everything until it’s done, then move to the next one and repeat?

SamMarrocco

Jay,
That all sounds good for Legend. If I might make some suggestions…..

While that “grouping bugs by feature…in this thread” link seems like a good idea, it’s far too convoluted for tracking. Also, web forum threads are notoriously difficult to follow (although not as horrific as Discord and the like). You spend more time navigating the web page than actually reading.

If you’re going to ‘present’ bug/feature information, I think you need something other than the forums as a giant pile of threads with replies all over the place. Bugs should not be anywhere near new features in those threads. It may make a good to-do list for you, but they have nothing to do with each other. Serious bugs are always priority 1, and supersede all other items.

Regarding what you said about “I think it could be good to pick a feature that needs work and just finish everything until it’s done, then move to the next one and repeat?”…..

I agree with what you said about finishing a feature until it’s done before moving on to the next feature, but all work on features should halt immediately once a major bug is found so that all effort is dedicated to fixing the bug. I realize that the problem with this is that new features can be left in various states of progress and stability……but this can be solved by a longer test period for new features before release, with a longer ‘in-house’ testing period before they are hoisted upon the public testers or general public.

My experience has been that the only beta testers truly worth having are the ones that work for you. Why? Because their compensation and job lifetime is based upon their performance. The man at the top releases the code to the public when he knows his employee testers have done their job–if not, it’s time to find new employees. I know that sounds harsh, but it’s the reason good development takes time and money. Otherwise, lame testers are kept because they don’t cost anything, and this is a race to the bottom for quality.

Using this logic, even I qualify as a ‘lame beta tester’ for the following reasons with Legend…. My only interest is purely selfish: I want a replacement for Workflowy/Dynalist that has most of it’s features, 100% reliability and gives me Panes, all before Workflowy adds Panes as a new features (which they’ve been saying is ‘on the list’ for years). This means that I have a ‘narrow focus’ for what I want to see work properly. since I already have a full-time job (and too many hobbies) my time to ‘test’ is limited to my personal interest–although I would like to see the program succeed for it’s own sake.

A good example of ‘focused or concentrated interest’…..I was a bit insulted by a Workflowy developer’s comments one day a few weeks ago regarding slow development, so in typical human fashion, I thought “I’ll show them, I’m going to try switch to another outliner! Grrrr!” So I spent a few long evenings back in Notion, Legend, and all the others, funneling down back to Legend, thinking I’ll give it another go. I hit all the synchronization bugs, and ended up back to Workflowy. Boom, my interested in legend abruptly ended again, although I did try my best to document everything I had ‘discovered’ regarding the sync bugs.

Jerud

Jorge It’s not OCD… it’s’just practical reasons….

You’re right of course, I was somewhat making light of the subject, in an attempt to do something other than rant for paragraphs, which is my usual MO : )

Back to the original thread topic, I’m wondering if there’s anything further we can do to assist @Jay in solving these sync issues? I’ve been working directly from desktop most of the last year due to my circumstances, but I did use the mobile app yesterday to knock out a whole bunch of work, only to find that NONE of those changes synced back to desktop. The items themselves were relatively new – I had just planned the work out Sunday night. The mobile app definitely picked up the new items Monday morning (you know, after force-quitting the app and relaunching, which I have to do any time the iOS app has been backgrounded for more than a couple hours). I worked off mobile all day Monday, then sat down this AM on desktop to review today’s work and found my same list from Sunday night. I did the full relaunch/refresh dance and no change.

I had switched the desktop app to stable when board folders broke, so I thought maybe I should flip my mobile to stable, too. Now my mobile agenda contains 8 overdue/ incomplete items from 2021 that I’m pretty confident I either completed or deleted. This includes one item which is in fact complete and displays in ‘complete’ color, even though the Pane is filtered to hide completes. When I check my agenda Pane on desktop, none of these items are present. I’ve also got at least 4 zombie Documents appearing in my Overview drawer.

Jay

As I mentioned in another thread, we’ve been working on a full rewrite of the sync system which should fix any remaining sync issues. And once it’s released into Legend, we’ll open source it so that others can use it and potentially find issues with it. We’ve already open sourced most of it and will add the new sync layer once the Legend update using it is out.

We’re also planning to make it save with Markdown files in the desktop app, as well as saving Boards and settings to files, so you could easily save your own backups.

I appreciate the offer to assist in solving these issues! We will hopefully release the new sync system before the end of the year, and then I will very happily appreciate your testing with it 🙂.