Saving (and loading) an initialized factory

[tinleym]

Is there a good way to save an initialized factory to file to load later? I'd like to be able to start up a bigger factory faster.

[kettanaito]

Hey, @tinleym.

There's no designated API for that. We are implementing what effectively is serialization/deserialization of entities in #87 but I can't give any promises that are going to be a public API you can utilize on your own.

Are you concerned with the runtime or compile-time performance? If it's the former, consider using memorization and other similar tools to reduce the cost of generating a large number of entities. I don't think you can achieve much compile-time performance as the main load happens internally in the library (iteration over model properties, seeding their values, defining the relations).

The most important question I'd ask is: what kind of performance issues you are experiencing? Do you have some metrics to share?

We have some performance tests in the repo and the current benchmark is 1000 entities in ~80ms. It's worth mentioning that the metrics are synthetic and use a rather plain model definition (no relations, for example). That should still be a fairly good default. I'd like to learn more from you about the performance issues you're experiencing.

[tinleym]

Thank you for your thoughtful answer (and your work in general, I really like this library and msw)!

I wouldn't exactly call what I'm experiencing an issue; I think this library handles reasonable demands rather well. My question comes more from the angle of "how big can I make my store if I don't want to exceed x seconds of loading?" The lag that prompted my question comes from initializing a network of relationships via multi-level nested loops... make n big enough, it'll take some time to build. In my experience, the store is pretty snappy once initialization is done.

The solution for my current development use case is simple enough, chop down n. However, I think one of the potential killer applications of msw is as a demo/mock up backend... I can hand over a functional mock data-intensive app to someone, and they can experience it in full stride. Here, I'd like to be able to make n as big as possible within a given initialization time, and that's where I hypothesized serialization might offer a boost, but memoization is a good suggestion -- and I'd be grateful for any others :).

I'm happy to set up a sandbox to keep score of n and performance... maybe a form that prompts the user to add nesting levels w/ an n for each level, a submit, and an interval output?

[kettanaito]

There's definitely room for improvement internally but I don't prioritize such work until we have actual performance degradation reports. Our current implementation is rather straightforward and I wouldn't sacrifice that over premature optimization at the moment (since we have no reports). Complex models and a big number of entities will, inevitably, take more time to bootstrap. It also comes down to those two criteria when modeling the data. I'd say most applications don't have this kind of complexity but I may be wrong.