Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Automatically provide cursor filter methods for all schema fields; also provide clean AJAX solution for pieces pages #766

Merged
merged 10 commits into from
Dec 1, 2016
Merged

Automatically provide cursor filter methods for all schema fields; also provide clean AJAX solution for pieces pages #766

merged 10 commits into from
Dec 1, 2016

Conversation

boutell
Copy link
Member

@boutell boutell commented Nov 30, 2016 •
edited
Loading

FILTERS

Most schema field types now automatically get cursor filter methods by the same name. This allows you to write, for instance:

self.find(req)._departments('some-event-id').toArray(...)

... To retrieve only people who are joined with departments (such a join must exist in the schema for people).

For joins you actually get several filters:

departments: expects slugs. Singular, or an array. Notice the lack of an _ makes this suitable for the public to see in a query string, and it also expects a slug which is what you expect in a query string (hostile URLs aren't cool).
_departments: expects ids. Singular, or an array. Notice the _ is what our devs already expect with a join and it takes ids, which are what devs usually prefer to pass (they never change).
departmentsAnd: like departments, but if you pass an array of slugs, they ALL must appear. With regular departments, if you pass an array, it's good enough if ANY appear. (The latter is the more common case and usually better UX, so it gets the default name.)
_departmentsAnd: like departmentsAnd, but taking IDs.

All of these filters have a launder method but are by default marked safeFor: "manage" because it's not cool to assume it's safe to let people discover stuff about your data. However there is a new safeFilters method of cursors which can be used to conveniently mark an array of filter names as safeFor: "public".

On the front end, apostrophe-pieces-pages now leverages this to implement the new piecesFilters option. An example from a client project:

  piecesFilters: [
    { name: 'system' },
    { name: 'year' },
    { name: 'tags'  }
  ]

This automatically populates req.data.piecesFilters.system with an array of choices to select a biological system (via a joinByOne), req.data.piecesFilters.year with an array of choices for years, etc.

Note that the join field in the schema is actually named _system; as mentioned above, the filter without the underscore expects slugs, and provides slugs as values for choices.

The choices are objects in our standard "label and value properties" format.

The choices are case-insensitively sorted.

This eliminates the need for any custom code for pieces and pieces-pages in many common cases.

This is all unit tested.

PRESENT LIMITATIONS:

  • More capabilities are coming such as specifying that you want autocomplete rather than an array of choices, which could in some cases be too much of a performance hit. The configuration format is designed to accommodate more properties to cover this.

  • There is no support for filtering on reverse joins yet. There are significant issues there, but we should dig into it.

AJAX

Implemented a general-purpose, extremely easy solution for AJAX filtering with bookmarkable/shareable URLs and correct browser history behavior. Add a data-apos-ajax-context="name" attribute to the outer div that should be refreshed when any link or form submission inside it takes place and has a URL that points back to the same page. Boom that's it. Combine this with the filters features above and you can do remarkable things with zero code.

(The name of the context must be unique on the page, which is always the case for pieces index pages. We're not committing to implementing support for multiple filterable widgets on one page with bookmarkable URLs across all of them, but we're not slamming the door either.)

  • Text fields and textareas in the AJAX context are not replaced, so that submitting the form on keyboard events does not result in broken input behavior. Their ancestors are also not replaced.

OTHER

  • Fixed a bug in the refinalize feature of cursors. state.criteria is now cloned before finalize and restored after it. Otherwise many criteria are added twice after refinalize which causes a fatal error with a few, like text search in mongodb. This was needed to get the slug filters working right.

Tom Boutell added 8 commits November 20, 2016 13:59
automatic addition of cursor filters for all schema fields; initial s…
c4812ef 
…upport for browseFilters option for apostrophe-pieces-pages; 2x speedup of typical index pages even when this is not even being used
working filters for joinByArray and joinByOne with unit tests, just b…
c1d3cc1 
…efore I decided that 'and' is a separate filter for easier use of 'build'
simpler filter syntax for joins. Both can be simple values, the strin…
59abc76 
…g 'none', or arrays. For joinByArray you can also call '_catsAnd' rather than '_cats' to get an query rather than an query.
apos.ui.replaceUnpreserved method for painlessly implementing AJAX re…
12442e5 
…freshes of forms without accidentally clobbering text input in progress
* Unit tested the select filters and fixed bugs along the way
bf61eec 
* `browseFilters` is now `piecesFilters`, which addresses both Alex's concern about imposing a UI and my concern about potential conflict with other meanings of the term `filters` that could apply to `apostrophe-pieces-pages` in its role as a manager of its own page type
Merge branch 'master' into browse-filters
6c2bf0d
* Fixed schema filter sanitizers
ec7e725 
* Made the schema filters safeFor: 'manage' by default, added `safeFilters()` convenience method to cursors, called it from `apostrophe-pieces-pages` when `piecesFilters` are in play because there's no other reason to call `indexCursor`
* The option to apostrophe-pieces-pages to declare some of the filters public and load choices for them for browsing purposes is officially and finally named `piecesFilters`, I will fight you (long story)
Initial reference doc of piecesFilters option, this needs to get mi…
0fbdbc9 
…grated to a HOWTO but we also need to think about whether we want to kill here() macros and built all this off old-school forms that have a little bit of progressive enhancement when desired instead, because they solve certain problems better
bgantick
bgantick reviewed Dec 1, 2016
View reviewed changes
Copy link
Contributor

@bgantick bgantick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing to look at. Looks good otherwise!

lib/modules/apostrophe-pieces-pages/index.js
@@ -78,6 +89,7 @@ module.exports = {

self.indexCursor = function(req) {
return self.pieces.find(req, {})
.safeFilters(_.pluck(self.piecesFilters, 'name'))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI - looks like _.pluck() was removed from lodash in 4.x: https://github.com/lodash/lodash/wiki/Changelog#v400

Maybe use _.map() instead?

@boutell
Copy link
Member Author

boutell commented Dec 1, 2016 via email

@boutell
Copy link
Member Author

boutell commented Dec 1, 2016 via email

* Implemented a general-purpose solution for AJAX filtering with book…
38effa3 
…markable/shareable URLs and correct browser history behavior. Add a `data-apos-ajax-context="name"` attribute to the outer div that should be refreshed when any link or form submission inside it takes place and has a URL that points back to the same page. Boom that's it. The `name` must be unique on the page, which is always the case for pieces index pages, but this provision is made as a starting point toward allowing multiple AJAX environments on the page for widget filters. Doing so would require considerable thought toward using nonconflicting query parameters and passing others on. It may never turn out to be a wise idea, we just don't want to slam the door on it.

* Text fields and textareas in the AJAX context are *not* replaced, so that submitting the form on keyboard events does not result in broken input behavior. Their ancestors are also not replaced.
* Implemented new slug-based join filters. The slug-based versions do *not* take the leading `_` in their names, which is perfect because they are seen by the public in query strings. Programmers will expect the versions with the `_` and pass ids to them and get what they expect. Nerdvana.
* Unit tests for the slug-based join filters.
* Fixed a bug in the `refinalize` feature of cursors. state.criteria is now cloned before finalize and restored after it. Otherwise many criteria are added twice after refinalize which causes a fatal error with a few, like text search in mongodb.
* Refactored and cleaned up my code for generating join filter choices to be clearer and less redundant.
@boutell
Copy link
Member Author

boutell commented Dec 1, 2016

This PR now also covers our AJAX issue, both #625 and #626 would be resolved by merging it. Whoah.

@agilbert
Copy link
Member

agilbert commented Dec 1, 2016

Whoah.

@boutell
Copy link
Member Author

boutell commented Dec 1, 2016 via email

@boutell boutell changed the title Automatically provide cursor filter methods for all schema fields Automatically provide cursor filter methods for all schema fields; also provide clean AJAX solution for pieces pages Dec 1, 2016
@boutell boutell merged commit cf93d15 into master Dec 1, 2016
This was referenced Dec 1, 2016
Closed
Closed
boutell pushed a commit that referenced this pull request Dec 1, 2016
2.9.0
0f3d7d5 
All tests passing.

* Two major new features in this release: built-in filters for most schema fields, and built-in AJAX support for `apostrophe-pieces-pages`. These combine to eliminate the need for custom code in a wide array of situations where you wish to allow users to browse and filter blog posts, events, etc. In most cases there is no longer any need to write your own `cursor.js` or your own AJAX implementation. The provided AJAX implementation handles browser history operations, bookmarking and sharing properly and is SEO-friendly.

[See the official summary of the pull request for details and examples of usage.](#766)

* We also fixed a bug in the `refinalize` feature of cursors. state.criteria is now cloned before finalize and restored after it. Otherwise many criteria are added twice after refinalize which causes a fatal error with a few, like text search in mongodb.

In addition, we merged a contribution from Fotis Paraskevopoulos that allows a `bodyParser` option with `json` and `urlencoded` properties to be passed to the `apostrophe-express` module. Those properties are passed on to configure those two body parser middleware functions.
@bgantick bgantick deleted the browse-filters branch September 11, 2017 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@bgantick bgantick bgantick left review comments

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@boutell @agilbert @bgantick