I recently came across a cloud based system which used postgres, mongodb & elasticsearch. The per service DB is followed, to the extent that each service was running its own cluster of storage engine which hosts only one database.
In default installations, the configuration file is at /etc/mongod.conf. What the MongoDB docs don't mention is that when migrating to WiredTiger we also need to update the configuration file to the new YAML format introduced in 2.6.As far as I can tell the engine option is only available in the new configuration format.Migrating from the old storage engine consists in creating a database dump, shutting down mongodb, changing settings and then importing the dump into the new storage engine.
So, what happens when your application fails while an embedded engine is running The net effect is that the application will likely receive some source records after restart that it had already processed right before the crash. How many depends upon how frequently the engine flushes offsets to its store (via the offset.flush.interval.ms property) and how many source records the specific connector returns in one batch. The best case is that the offsets are flushed every time (e.g., offset.flush.interval.ms is set to 0), but even then the embedded engine will still only flush the offsets after each batch of source records is received from the connector. 153554b96e