New filtering

Jay

Jorge I’m having trouble reproducing that first one. In that example Child 2 is the only text match but it’s bringing in Child 1 also? Do any other children of Parent A also match incorrectly? Do you have any other filters enabled besides just text?

I see the List problems happening with no filters (it works fine with “hide completed”), will work on that.

Jorge

Jay Yes that’s exactly right. Child 2 is the only one that matches the text filter but it also brings Child 1 as well.
There were no other children of Parent A, but there were other Parents within the 19 results that had children that didn’t match the filter and they were correctly not being shown. The only odd thing that I noticed is that Child 1 was a destination of links (had a tally box of 3 at the right). I don’t think I had any other filters enabled because most of the items I was looking at were completed, so I had the complete filter show all complete and incomplete items.

Jay

LauraH I think that should be fixed in the latest update. And Jorge you mentioned items missing in List view which I think should be fixed too.

LauraH

Jay Something still not right…

Jay

Jorge Is it fixed in the latest update? I did fix some issues with mirrors appearing when they shouldn’t.

Jay

LauraH This one should be fixed in the latest version.

LauraH I can’t figure out how to reproduce this one. Can you show send me a screenshot of what the hierarchy looks like for both where the mirror is and where the origin item is?

LauraH

My Understanding (the way I think it’s always been?) - “List View” will show only matches, but will show them from anywhere within the selected outline node - no matter how deep they may be buried. Something in the recent updates broke this.

I have a “view by project” pane, “Show from anywhere in the hierarchy” is selected.
I have a couple text filters set also.
When I view this in OUTLINE mode - it correctly shows matches from deep in the hierarchy (I have to expand to see them, but that is expected in OUTLINE mode.)
When I view the exact same thing in LIST mode - anything more than one child deep (directly under the project) disappears.

I EXPECT List view to show me just the matches, but show them no matter where they are - that’s the way it’s been and the way I’ve always used this pane! But having my deeper items not show in list view is messing me up!

LauraH

Jay

Screenshots sent via DM Jay.
Possibly due to the fact that I completed the PARENT of the “Think through..” task, but not that individual task.

LauraH

Another filter bug: Similar to the above - seems related to an item being a MIRROR, and the parent task being completed, but not the mirrored task itself. Note the parent of “Blondies” (Thursday, 20 May 2021) does have a date of 20 May. So likely why it’s triggering the “Overdue” flag?

LauraH

Another Mirror Filtering Bug:
This task (Thaw Chicken) is mirrored into my “Today” list and should show on the list filtered to show tasks… but it’s not there.

I know you’re working on it; but these need to be solved Jay. If mirrors do not filter correctly; they become worse than problematic/buggy - they become a liability. I put something somewhere and expect I will then see it on my relevant lists… but then it does not show as expected I am missing key information. I keep wanting to use mirrors in my system as they have so much promise and potential - but behavior like this means I cannot trust them… and therefore don’t dare actually use them.

Jay

The latest update is another full rewrite of the filtering algorithm along with how pane items are generated. The previously new algorithm was not able to support a key text filtering feature: an item matching if the filter text is contained by the combination of an item’s full hierarchy. For example, these items should match on a filter for “a b c”

a
- b
  - c

Similar to that it was not handling hierarchy around mirrors correctly. An item’s matchingnessitude comes from its whole hierarchy - a combination of itself, its parents, and its children. An item is considered matched if its parent matches, unless it itself specifically does not match. Or a parent is considered matched if any descendant matches. But when computing the parent matches for mirror items it was traversing up the target item’s tree instead of the mirror item. Part of the problem is that filtering was originally designed to compute matches by item, but with mirrors an item can match in one area of the hierarchy but not in another, because their different ancestors cause them to match differently.

So I rewrote the whole thing from scratch in a new way that takes an item’s full hierarchy into account (without needing to test each one more than once) to fix both of those issues. It turned out to be even shorter and less complex, and even removed a step of the process of generating pane items:

Filter items to find matches
Scan through all items and check matches to build list of pane items (removed, now uses results of #1)
Iterate pane items and group into blocks
Sort blocks and generate display elements

So it should also be a bit faster now with that step removed 🙂.

This may additionally fix some of the bugs reported in this thread as well as in https://forum.legendapp.com/d/125-mirrors-bug-mega-compilation-from-slack. I will go back and try to reproduce the previously reported issues but if you can re-test the bugs you found and let me know their status that would help a lot!

Sorry for the flurry of changes around filtering recently! I’m trying to really get this right and lock down filtering as perfectly correct and stable. It’s extremely complex with how it deals with hierarchy, and mirrors throw a big wrench in the hierarchy 😆. I’ve built up a big suite of automated tests so in addition to lots of manual testing I’m pretty confident it’s correct now. But please let me know if you see any more issues (including screenshots is very helpful) and I’ll add more tests and fix them 👍️

andrew

Jay Big problem with the new filtering is that the outline is not auto-expanding to show items matching the text filter when entered.

Jay

andrew Oh thanks for catching that! I had forgotten to copy that over from the old algorithm. It should be good now in the latest beta.

LauraH

I’ve still got something wrong with completed mirrored items showing up when they shouldn’t.

andrew

When I apply a filter (like star) in one pane, it will include a matching item that is a descendant of an item that is a mirror of an item in another document. Then when I edit the matching item in another pane to remove the star, the item disappears in the first pane, as it should. BUT when I reapply star to the same item in that other pane such that it should now match the filter in the first pane, it does not reappear in the first pane unless I unapply and reapply the filter in the first pane (or sometimes after I switch away from the app and then back).

Jerud

Something falls apart in Group-By Tag Panes. I will describe this in terms of my application because it’s hard for me to generalize in a meaningful way:

I distinguish tags for geographical places by beginning them with @. So: #@UT-SLC is for salt lake city (BTW wouldn’t it be cool to have geo-tags?).
These #@ tags are applied to individual tasks but also to parents and Projects.
Of course I have regular #tags as well
I want to show only geographical places in a group-by-tags Pane. So I apply the text filter: “#@”
Many of the #tag groups are removed, as expected – but not all of them
The #tag groups which remain are due to this breadcrumb-matching. If an item contains a #tag and is also under a parent which has a #@ tag, it matches the text filter but gets grouped up under each of the #tags that appear in its complete breadcrumbs. So my Pane must include a tag group for each of those #tags, which are what I was trying to filter out in the first place. This item also appears, of course, under the #@ tag heading as a child – which is the only place I wanted it.

The opposite problem happens with negated filters. In my example above, if I want to view only tags that aren’t #@ geo tags I create the filter “-#@”. But this results in a Pane with nearly no content at all, since most of my items are under geo-tagged parents and thus excluded from the results.

Meanwhile, in non-grouped Panes filtering seems to behave as it always did. This confuses me because it actually seems like the issues I describe above aren’t bugs – they sound exactly like what Jay describes with breadcrumb matching. But then, the fact that my other filtered Panes are acting like they always have, makes me think I’m not really understanding. Maybe I’m just tired.

Oh, also: I can’t expand/collapse parents within group-by-Pane groups. The groups themselves can open/close but not any of their contents. I think this is a recent development connected to the filter rewrite.