RavenDB 3.0. New Stable Release

We have just released build 3785 of RavenDB 3.0. This build has quite a bit of changes (for the full gory details, see the pull request).

This release includes about 3 months of bug fixes, performance improvements and the like. We have been testing this on our own systems for a few weeks now, as well as on multiple live production sites, and the results have been nothing but encouraging.

Major changes

• Voron & map/reduce optimizations. We have done major work to optimize how RavenDB uses map/reduce on Voron. As a result, map/reduce performance on Voron has improved tremendously. However, this require a migration step during the first startup. If you have a large RavenDB database using Voron, and you are making heavy use of map/reduce, take into account that on first start, RavenDB will need to perform an internal migration, which can take a while.
• Lucene & memory allocation reduction on queries. We have drastically reduced the amount of memory that is allocated per query, and improved the performance of queries substantially.

Improvements:

• Many small perf optimizations, memory allocations reductions, object pooling, etc. Drastic reduction in memory allocations on common code paths.
• Better handling of buffer allocations in websockets, reduces memory fragmentation.
• Better handling of Take() / Skip() inside an index.
• Allow only a single index to use the fast precomputation optimization at a time (reduce memory usage if multiple medium sized indexes are changed concurrently).
• Better handling of concurrent addition of multiple indexes to large databases, will now run in the same set of indexing batches, instead of each having their own.
• Re-implemented memory statistics checks using native calls to avoid expensive allocations.
• Provide more detailed information when an index is corrupted.
• Adding endpoint for stopping / starting just reduce work.
• Less aggressive changes to the batch size at scale, being more cautious gives us a bit slower perf but more stable system under load.
• Optimized Voron recovery code heavily to support slow I/O systems on large databases.
• Allow to mark individual databases as development / staging /production.
• Better handling of Lucene file usage, using mmap to avoid all allocations when querying the indexes. Significant improvements to both memory usage and querying speeds.
• Subscriptions can now start from a given etag.
• Subscriptions now offer more robust handling for attempting to open an existing subscription.
• Subscriptions will now send a “no results found” so we won’t time out for mostly idle subscriptions.
• Allow to manage scripted index scripts from code using AbstractScriptedIndexCreationTask.
• Fixing replication issue with RavenFS with large number of files being modified all the time.
• Track the query parse time as well, for certain queries the expensive part is parsing the query, rather than executing it.
• Can lock transformers for modifications now as well as indexes.
• Better heuristics for calculating how much memory (native & managed) we are actually using.
• Add debug endpoint to track how much map/reduce work we still have to do.

Bug Fixes:

• Don’t update a side by side index if it already exits.
• Allow to update a side by side index while it is still running.
• Fixing index compilation error on .NET 4.6 using “new string[0]”.
• Fixed an NRE when the index definition was removed forcibly when using dynamic queries.
• Fixed error handling during disposal causing an exception to escape thread boundary and crashing.
• Fixed FIPS licensing issue on embedded dbs.
• Admin logs are not capturing logs protected by IsDebugEnabled statement
• Fixed a finalizer usage bug causing us to try to read from a closed handle.
• Prevent corrupted index warning when creating a map-reduce index and indexing is disabled.
• Preventing code from trying to use disposed internal transactions.
• Installed fix – check and revoke URL reservation options when Use existing website is selected.
• Properly dispose of timer instance when shutting down a database using expiration bundle.
• Prevent an error loading ICSharpCode.NRefactory from killing RavenDB client startup.
• When disk space is very low, stop indexing and warn about it, rather than index to full disk error (and probable index corruption).
• Fixing stack trace generation in generate debug info when we have spaces in the temp path.
• Moved default db locations outside of the IIS directory to avoid IIS bug causing restarts.
• When out of memory, replication will back off and retry, rather than fail continuously.
• Better handling of deleted then created indexes and transformer replicating to sibling nodes.
• Fixing case sensitivity issue when returning document ids.