The long battle of upgrading Discourse's Javascript from constants to ES6 modules finally has an end in sight. The vast majority of Javascript files we use to serve up Discourse have been converted to ES6 modules and are working great in production.
Having said that, there are some holdouts in our code base that have not been converted yet, mainly because they are difficult. In particular, our Model/data fetching infrastructure hasn't been upgraded in years and has a lot of room for improvement.
The Old Way to fetch data
Right now, the standard way to fetch data is to perform a Discourse.ajax call, then to wrap the result in an instance of the model you want. For example:
Widget = Ember.Object.extend();
Widget.reopenClass({
find(id) {
return Discourse.ajax("/widget/" + id + ".json").then((result) => {
return Widget.create(result);
});
}
});
With the above code, you could say Widget.find(123) and get a promise that will resolve into a Widget.
This approach is not bad, it has obviously scaled up to our needs so far and kept our data layer simple. However, there is a lot of room for improvement:
We repeat ourselves a lot. On the server side we use a REST convention but the client side ends up making the same kinds of calls a lot.
For more complex objects (that contain side loaded or embedded objects) the finders become more complex.
It has no concept of an identity map, introducing potential bugs as the same object can be loaded twice in memory.
The New way to fetch Data
The new Discourse betas have a store injected into routes, controllers and models. If you've used ember-data in the past this concept is familiar to you and that is no coincidence. The new API is similar to ember data but is based off Discourse's needs and is considerably simpler. It can be seen as us stealing many of the ideas of ember data but without going full hog down that road.
I've been slowly building up the store over the last few features I've built and the API is finally stable and good enough to use so I recommend it for all new features.
The core idea is you ask the store for data. For example, to find the widget in the example about you'd do this instead:
this.store.find('pet', 123);
The above code will automatically make a GET request to /pets/123.json. You don't have to write any more code in Discourse to do this. You don't have to define a pet model or hardcode the pets path or anything like that. The store will make the request, find the pet in the returned JSON and return it to you as an instance of RestModel.
RestModel is the new base class for our models. Since we didn't define a Pet model, we'll just get an instance of that. However, if we did want to return an instace of Pet instead we could define a class like this.
// discourse/models/pet.js.es6
import RestModel from 'discourse/models/rest';
export default RestModel.extend({
bark() {
console.log('woof');
}
});
If a Pet model is defined, the store will automatically instantiate it for you. If not, no bother.
RestModel instances come with some handy methods. For example, let's say you want to update a Pet's name. You could do this:
pet.update({ name: 'rover' });
That will automatically make a PUT request to /pets/:pet_id with the name attribute. Again this is based on Discoruse's conventions. (If you write your endpoints in our conventional way, you have to add zero client side code to handle this update.)
Result Sets
To find more than one of a thing, you can call this.store.findAll('pet'). This will make a GET to /pets.json and instantiate records for each Pet. You can also call this.store.find('pet', {dog: true}) and it will call /pets.json?dog=true as a filter.
Any time you get back multiple records from the store, it will look like a array, but it's actually a ResultSet. For almost all your code the difference doesn't matter, you can still loop through it and use all the Ember array functions. However, it has some added functionality to make our lives easier.
One thing we do a LOT is loading more data with scrolling. We used to have to code this over and over, so I made a common API for it. If a JSON result includes total_rows_pets and load_more_pets (the later is a URL to load more), the ResultSet will remember those attributes.
You can then just call model.loadMore() to load more results and it will call the remote URL and fetch more records. This means you have to write a lot less code.
RestModel States
We have a lot of repeated code that handles the cases of whether a model is currently saving. You'll see a lot of code that sets a saving property while a promise is resolving, then updating it to false when it finishes. With RestModel this is done automatically. While you are making the above update call, it will automatically set isSaving to true on the model. So if your template looks for that, it will be true while saving and false when it isn't. This should cut out a LOT of code for us.
Creating and Destroying Records
If you want to make a new instance of something that's easy:
const pet = this.store.createRecord('pet', {name: 'woofie'});
At that point, it will have isNew set to true. You can bind it to a form or do whatever you need to input data for the new model. When you are ready to save it, just call pet.save() and it will be sent across the wire. When a record is created, it makes a POST to /pets with the data, just like our conventions.
To destroy a record, just call pet.destroyRecord() and you're good to go. It will make a DELETE to /pets/:pet_id.
Loading Sideloaded Objects
Often you want to return a list of items that include associated data. For example our QueuedPostSerializer embeds a user and a topic. By default, ActiveModel::Serializers will sideload this data, which means that instead of every queued_post in the json having an embedded user and topic property, it will have a user_id and topic_id, and then separate users and topics collections on the root of the JSON that contains those objects. This can cut down on the JSON size quite a bit, but is difficult to instantiate on the client side.
With the new store code, there is a way to do this automatically! When serializing, add the rest_serializer: true option:
render_serialized(@queued_posts, QueuedPostSerializer, root: :queued_posts, rest_serializer: true)
If you do this, this.store.find('queued-posts') will automatically load and instantiate the sideloaded data! Any key that begins with something_id will look for a somethings collection in the root and instantiate Something models if they exist.
(As a bonus, I've added it so that if you include category_id anywhere in a rest_serializer: true dump, the category will automatically be found. We always have the categories loaded client side so there is no need to serialize anything but the id).
Identity Map
Anything found via the store will use an identity map. So if you retrieve the same user by id twice, you are guaranteed to be pointing at the same instance in memory. Updating it in one place will update it everywhere that it is bound. Note this is true even if the user comes back in a collection or side loaded as explained above.
Munging Data
Sometimes the data that comes back from the server isn't right for you. I'd highly suggest trying to use the rest_serializer: true option but if that can't work for you, you can just implement the munge() method. For example:
const Pet = RestModel.extend();
Pet.reopenClass({
munge(json) {
// weight comes back as lbs but we want kgs
json.weight = json.weight * 0.453592;
return json;
}
});
If a munge method is present JSON retrieved from the server will be passed through it before instantiated so you can modify it. Again, try not to do this unless dealing with old models or code.
Adapters
What if your JSON endpoints don't follow our RESTful conventions? You should definitely try to make everything follow the discourse conventions, but if you can't do that there is a way to modify where data is sent/retrieved.
If you define an adapter in discourse/adapters, it will be used to contact the server instead of the default REST method. For example, I implemented one of these for topic-list because it uses the PreloadStore to fetch data quickly. It will first look for the model in the PreloadStore and if it doesn't exist will use a custom finder.
Adapters are useful for when you don't control the JSON endpoint or the JSON endpoint doesn't work with our conventions. Most of the time going forward you shouldn't need to use them, but they are there if you need to do something different.
Conclusion
I've been using this store and RestModel in the last few features I've written and it has really made a huge difference in the amount of code I've written. The API is pretty good now, but I would love suggestions/tweaks if people have them.
does this make sense? what and how does a topic classified as a top referred topic..
