MikroORM 5.8 released

After a longer pause, I am pleased to announce next feature release - MikroORM v5.8, probably the last one before v6. While I don't blog about the feature releases very often, I feel like this one deserves more attention. Why?

Fine-grained control over upserting

Upserting, so em.upsert and em.upsertMany, is relatively new addition to the EntityManager toolkit. It allows you to run insert on conflict queries (and works in MongoDB too!) to ensure a record exists in the database, updating its state when necessary.

The default behavior was to always prefer using the primary key for the on conflict condition, and fallback to the first unique property with a value. While this was a sane default, it quickly appeared it won't work very well for some use cases - the most common one being entities with UUID primary keys. In this case, we cannot use the primary key for the conflict resolution - but we often generate this value via property initializers, so the default logic would prefer it (and fail to find a match). A way around this was using the POJO signature of em.upsert, and not providing the UUID value in the data - but that leads to counter issues with drivers that don't have an easy way to generate UUID on server side (via database default). And it is not just about drivers like SQLite, where we don't have a UUID function on database level - the very same problem comes with PostgreSQL too, when you want to generate a UUID v7 key - those are not yet supported out of box.

MikroORM v5.8 now offers a fine-grained control over how the upserting logic works. You can use the following advanced options now:

• onConflictFields?: (keyof T)[] to control the conflict clause
• onConflictAction?: 'ignore' | 'merge' used ignore and merge as that is how the QB methods are called
• onConflictMergeFields?: (keyof T)[] to control the merge clause
• onConflictExcludeFields?: (keyof T)[] to omit fields from the merge clause

Here is a more complex example, note that it would work with both signatures the same way - we can pass in an array of entities too, and leverage the runtime defaults we set via property initializers.

const [author1, author2, author3] = await em.upsertMany(Author, [{ ... }, { ... }, { ... }], {
  onConflictFields: ['email'], // specify a manual set of fields pass to the on conflict clause
  onConflictAction: 'merge',
  onConflictExcludeFields: ['id'],
});

This will generate query similar to the following:

insert into "author" 
  ("id", "current_age", "email", "foo")
  values
    (1, 41, 'a1', true),
    (2, 42, 'a2', true),
    (5, 43, 'a3', true)
  on conflict ("email") 
  do update set
    "current_age" = excluded."current_age",
    "foo" = excluded."foo" 
  returning "_id", "current_age", "foo", "bar"

It might not be obvious on the first sight, but since we specified onConflictExcludeFields: ['id'] in this example, the primary key value will be narrowed via the returning clause - in the third item, we specify a wrong primary key (5) which is not matching the database, and as a result, we get a property hydrated entity with the correct primary key we get from the database.

There were other hidden improvements made to the upserting, namely the qb.onConflict() now maps property names to column names properly, the em.upsertMany runs in batches now (respecting the global batchSize option), and the result of em.upsert/Many now contains all the values with database defaults, so it truly represents the current database state and not just the data provided by you in the payload.

Filters with Joined loading strategy

Another important change is about filters. Previously, the filters were applied only when using the select-in strategy, as part of the where condition. Since v5.8, when using the joined strategy, filters will be also applied to the query, via join on conditions. This means the strategies should now behave the same finally. I am currently considering whether we should change the default strategy in v6, as this was the last big feature missing in the joined strategy implementation.

A connected change to this one is how the populateWhere option works - those conditions are now also applied as join on conditions when using the joined strategy.

New Collection helpers

Lastly, there are several new Collection methods, so its interface is more in-line with the Array prototype:

• find() to find the first matching element
• map() to map elements
• filter() to filter elements
• reduce() to map the array to a dictionary
• slice() to extract a portion of the elements

And few more convenience methods:

• exists() to check if a matching element exists
• isEmpty() to check if the collection is empty
• indexBy() to map the items to a dictionary, if there are more items with the same key, only the first one will be present.

The indexBy() can be pretty helpful, let's have a closer look at what it is capable of. If we specify only the first parameter, we get a map of the elements indexed by a given key:

// user.config is `Collection<Option>` where `Option` has a `key` and `value` props
const config = user.config.indexBy('key');
// {
//   option1: { id: 5, key: 'option1', value: 'Value 1' },
//   option2: { id: 12, key: 'option2', value: 'Value 2' },
//   option3: { id: 8, key: 'option3', value: 'Value 3' },
// }

The second parameter allows to map to a value directly instead of mapping to the target entity:

const config = user.config.indexBy('key', 'value');
// {
//   option1: 'Value 1',
//   option2: 'Value 2',
//   option3: 'Value 3',
// }

Smaller improvements worth mentioning

• readonly transactions (native feature of MySQL and PostgreSQL)
• comments and hint comments on SQL queries (e.g. for query optimizers and logging purposes)
• global disableIdentityMap and resultCache options
• Migration class now offers getEntityManager() helper which accepts the current transaction context

What about v6?

I mentioned in the beginning that this might be the final feature release of the 5.x branch, so where are we with the next major? There are still few refactors I want to undergo, before I consider it done, but most of them are already in progress and shouldn't be too hard to resolve.

• support for propagation with useDefineForClassFields: true
• rework Date mapping to work via mapped types
• rework hydration of STI entities to support shadowing of properties with same type

I was focusing on the 5.x branch recently, so the v6 development was a bit stalled, but this is going to change now. But there is one significant improvement that was merged into v6 just yesterday - the initial dataloader support for loading Collection and Reference properties!

It will still take some time, definitely weeks, maybe a few months, but worst case, we should see the next major released by the end of the 2023.

Getting started guide

I got one more thing to share. Maybe you missed it, but the work-in-progress Getting Started guide is now public. It's far away from complete, but it should already provide some value, especially to beginners. Be sure to check it out, and as always, feedback is very welcome!

https://github.com/mikro-orm/guide