Struct FeatherWriter 

pub struct FeatherWriter { /* private fields */ }

Manages multiple FeatherBuffers and handles encoding, rotation, and flushing to the object store.

The write() method is the single entry point for clients: they supply a data value (of generic type T) and the manager encodes it (using T’s metadata via EncodeToRecordBatch), routes it by CatalogPathPrefix, and writes it to the appropriate FileWriter. When a writer’s buffer is full or rotation criteria are met, its contents are flushed to the object store and it is replaced.

Implementations

impl FeatherWriter

pub fn new( base_path: String, store: Arc<dyn ObjectStore>, clock: Rc<RefCell<dyn Clock>>, rotation_config: RotationConfig, included_types: Option<HashSet<String>>, per_instrument_types: Option<HashSet<String>>, flush_interval_ms: Option<u64>, ) -> Self

Creates a new FeatherWriter instance.

pub async fn write<T>(&mut self, data: T) -> Result<(), Box<dyn Error>>

where T: EncodeToRecordBatch + CatalogPathPrefix + 'static,

Writes a single data value. This is the user entry point. The data is encoded into a RecordBatch and written to the appropriate FileWriter. If the writer’s buffer reaches capacity or meets rotation criteria (based on the rotation configuration), the FileWriter is flushed to the object store and replaced.

pub async fn write_batch<T>( &mut self, data: Vec<T>, ) -> Result<(), Box<dyn Error>>

where T: EncodeToRecordBatch + CatalogPathPrefix + 'static,

Writes a batch of data values as one or more RecordBatches.

Uses T::chunk_metadata to derive the file schema metadata. This protects types like OrderBookDelta from having their file metadata poisoned by a leading sentinel row (e.g. BookAction::Clear, which carries price_precision=0, size_precision=0).

Per-instrument types are partitioned by instrument so a mixed-instrument batch lands in the correct file for each instrument.

pub async fn flush(&mut self) -> Result<(), Box<dyn Error>>

Flushes all active FeatherBuffers by writing any remaining buffered bytes to the object store.

This is called automatically based on flush_interval_ms if configured, but can also be called manually by the client.

Note: In Rust, we use in-memory buffers. Flushing writes the current buffer to the object store and creates a new buffer for continued writing. This is different from Python which just flushes OS buffers.

pub async fn close(&mut self) -> Result<(), Box<dyn Error>>

Closes all writers by flushing and removing them.

After calling this, no further writes should be performed.

pub fn is_closed(&self) -> bool

Returns whether the writer has been closed (all writers cleared).

pub fn get_current_file_info(&self) -> HashMap<String, (u64, String)>

Returns information about the current files being written.

Each entry maps a writer key (type_str and optional instrument_id) to its current buffer size and file path.

pub fn get_next_rotation_time( &self, type_str: &str, instrument_id: Option<&str>, ) -> Option<UnixNanos>

Returns the next rotation time for a specific writer key, if set.

pub fn is_duplicate_event_id(&mut self, event_id: &UUID4) -> bool

Returns whether the given event ID has already been seen, adding it to the cache if new.

pub async fn write_data(&mut self, data: Data) -> Result<(), Box<dyn Error>>

Writes a Data enum value to the appropriate writer.

This is a convenience method that routes the Data enum to the appropriate typed write method.

pub async fn write_instrument( &mut self, instrument: InstrumentAny, ) -> Result<(), Box<dyn Error>>

Writes an instrument to the appropriate writer.

Instruments are written to feather files and organized by instrument ID. This method supports writing instruments that implement EncodeToRecordBatch and CatalogPathPrefix.

pub fn subscribe_to_message_bus( writer: Rc<RefCell<Self>>, ) -> Result<ShareableMessageHandler, Box<dyn Error>>

Subscribes to all messages on the message bus (pattern “*”).

This will automatically write all supported data types that are published on the message bus to the feather files.

The writer must be wrapped in Rc<RefCell<>> to be shareable with the message bus handler.

Note: The handler spawns async tasks to write data, so writes happen asynchronously and won’t block the message bus.

pub fn unsubscribe_from_message_bus(handler: &ShareableMessageHandler)

Unsubscribes from the message bus.