Struct FeatherWriter 

pub struct FeatherWriter { /* private fields */ }

Manages multiple FeatherBuffers and handles encoding, rotation, and flushing to the object store.

The write() method is the single entry point for clients: they supply a data value (of generic type T) and the manager encodes it (using T’s metadata via EncodeToRecordBatch), routes it by CatalogPathPrefix, and writes it to the appropriate FileWriter. When a writer’s buffer is full or rotation criteria are met, its contents are flushed to the object store and it is replaced.

Implementations

impl FeatherWriter

pub fn new( base_path: String, store: Arc<dyn ObjectStore>, clock: WriterClock, rotation_config: RotationConfig, included_types: Option<HashSet<String>>, per_instrument_types: Option<HashSet<String>>, flush_interval_ms: Option<u64>, ) -> Self

Creates a new FeatherWriter instance.

pub fn with_record_filter( self, record_filter: Option<WriterRecordFilter>, ) -> Self

Sets typed record-family filter for subsequent writes.

pub fn with_catalog_identifier_column(self) -> Self

Includes the catalog row identifier column in staged batches.

pub fn write<T>(&mut self, data: T) -> Result<(), Box<dyn Error>>

where T: EncodeToRecordBatch + CatalogPathPrefix + 'static,

Writes a single data value.

This is the user entry point. The data is encoded into a RecordBatch and written to the appropriate FileWriter. The encode path is fully synchronous; the runtime is entered only when a rotation or auto-flush boundary is actually hit.

pub fn write_batch<T>(&mut self, data: Vec<T>) -> Result<(), Box<dyn Error>>

where T: EncodeToRecordBatch + CatalogPathPrefix + 'static,

Writes a batch of data values as one or more RecordBatches.

Uses T::chunk_metadata to derive the file schema metadata. This protects types like OrderBookDelta from having their file metadata poisoned by a leading sentinel row (e.g. BookAction::Clear, which carries price_precision=0, size_precision=0).

Per-instrument types are partitioned by instrument so a mixed-instrument batch lands in the correct file for each instrument.

pub async fn flush(&mut self) -> Result<(), Box<dyn Error>>

Flushes all active FeatherBuffers by writing any remaining buffered bytes to the object store.

This is called automatically based on flush_interval_ms if configured, but can also be called manually by the client.

Note: In Rust, we use in-memory buffers. Flushing writes the current buffer to the object store and creates a new buffer for continued writing. This is different from Python which just flushes OS buffers.

pub async fn close(&mut self) -> Result<(), Box<dyn Error>>

Closes all writers by flushing and removing them.

After calling this, no further writes should be performed.

pub fn is_closed(&self) -> bool

Returns whether the writer has been closed (all writers cleared).

pub fn get_current_file_info(&self) -> HashMap<String, (u64, String)>

Returns information about the current files being written.

Each entry maps a writer key (type_str and optional instrument_id) to its current buffer size and file path.

pub fn buffered_totals(&self) -> (u64, u64)

Returns the total buffered (unflushed) bytes and rows across all active buffers.

pub fn get_next_rotation_time( &self, type_str: &str, instrument_id: Option<&str>, ) -> Option<UnixNanos>

Returns the next rotation time for a specific writer key, if set.

pub fn write_data(&mut self, data: Data) -> Result<(), Box<dyn Error>>

Writes a Data enum value to the appropriate writer.

This is a convenience method that routes the Data enum to the appropriate typed write method.

pub fn write_data_batch( &mut self, data: Vec<Data>, ) -> Result<(), Box<dyn Error>>

Writes mixed data as typed Arrow batches.

Panics

Panics if the built-in data batch dispatch becomes non-exhaustive.

pub fn write_any_message( &mut self, message: &dyn Any, ) -> Result<bool, Box<dyn Error>>

Writes supported message bus value.

pub fn write_instrument( &mut self, instrument: InstrumentAny, ) -> Result<(), Box<dyn Error>>

Writes an instrument to the appropriate writer.

Instruments are written to feather files and organized by instrument ID. This method supports writing instruments that implement EncodeToRecordBatch and CatalogPathPrefix.

pub fn subscribe_to_message_bus( writer: Rc<RefCell<Self>>, ) -> Result<StreamingSinkSubscription, Box<dyn Error>>

Subscribes to all messages on the message bus (pattern “*”).

This will automatically write all supported data types that are published on the message bus to the feather files.

The writer must be wrapped in Rc<RefCell<>> to be shareable with the message bus handler.

Note: Writes are synchronous; the writer enters the runtime internally only when a rotation or auto-flush boundary is hit.

pub fn unsubscribe_from_message_bus(handler: &StreamingSinkSubscription)

Unsubscribes from message bus.

Trait Implementations

impl Debug for FeatherWriter

fn fmt(&self, f: &mut Formatter<'_>) -> Result

Formats the value using the given formatter.

impl StreamingSink for FeatherWriter

fn write_data(&mut self, data: Data) -> Result<()>

Writes a single Data enum value.

fn write_batch(&mut self, data: Vec<Data>) -> Result<()>

Writes a batch of Data values.

fn write_any(&mut self, message: &dyn Any) -> Result<bool>

Writes any supported Nautilus message value.

fn flush(&mut self) -> Result<()>

Flushes buffered data to durable storage.

fn close(&mut self) -> Result<()>

Closes sink after flushing buffered data.