Log Data Model Changes and Disk Buffers
We’re bringing our log data model closer to JSON
We are currently working to improve and optimize our
model. Initial versions of this data model were
represented as a flat map for key access optimizations. This proved over time
to not be as helpful as we had hoped. As a result we are working to move our
data model to be as close to JSON as possible. This means:
nullvalues are now supported in Vector’s data model.
- Nested fields are represented in an actual nested representation.
Both of these changes bring Vector’s internal data model closer to JSON. Unfortunately, this breaks disk buffer serialization which means you must drain your disk buffer before upgrading Vector.
- Make sure Vector shuts down normally to ensure your disk buffers are fully drained.
- That’s it! Update Vector as usual.
Note, Vector will discard invalid disk buffer data, bad data will not prevent Vector from starting.