Allow configuration with chunk size when streaming #14

New issue

Open

opened 2026-06-08 01:51:55 +00:00 by linarphy · 2 comments

linarphy commented

2026-06-08 01:51:55 +00:00

Owner

Can be done using a configuration file, an environment variable, or a cli argument, but there is no cli yet.
For now, Converter don’t know about chunk, only the example class MermaidConverter use it. Should I specify it in the base class ?

Can be done using a configuration file, an environment variable, or a cli argument, but there is no cli yet. For now, Converter don’t know about chunk, only the example class MermaidConverter use it. Should I specify it in the base class ?

linarphy commented

2026-06-09 16:09:23 +00:00

Author

Owner

I think chunk size is a mandatory setting for the Converter base class:
A stream is a flow of data. A Graph can only contain a discrete number of components. . If we suppose any creation of Graph from a string (construction) is procedural and incremental, there must be a components (Node or Connection) which is the last one added in any constructed Graph. The last element is defined as the element that do not have an element after it. Meaning the content of the string after the element has to be checked (and read if it exists).

To know the end of a flow of data, one has to wait for it. In Python, that means fetching N bytes of data until no data is retrieved. This N number has to be defined. It can be hardcoded (not modifiable easily) or a configurable value.

The live and dead memory of a system are the parameters that guide the value of this number. As multiple systems exist, with different live and dead memory, this number must be configurable.

I think chunk size is a mandatory setting for the `Converter` base class: A stream is a flow of data. A `Graph` can only contain a discrete number of components. . If we suppose any creation of `Graph` from a string (*construction*) is procedural and incremental, there must be a components (Node or Connection) which is the last one added in any *constructed* `Graph`. The last element is defined as the element that do not have an element after it. Meaning the content of the string after the element has to be checked (and read if it exists). To know the end of a flow of data, one has to wait for it. In Python, that means fetching **N** bytes of data until no data is retrieved. This **N** number has to be defined. It can be hardcoded (not modifiable easily) or a configurable value. The live and dead memory of a system are the parameters that guide the value of this number. As multiple systems exist, with different live and dead memory, this number must be configurable.

linarphy added the

Importance/Normal

Complexity/Low

Kind/Feature

labels

2026-06-09 16:09:46 +00:00

linarphy self-assigned this

2026-06-09 16:09:53 +00:00

linarphy removed their assignment

2026-06-09 16:09:54 +00:00

linarphy commented

2026-06-09 16:22:28 +00:00

Author

Owner

I should make sure every subclass of Converter use this setting (even if they use a library to ease the implementation, they must use libraries that allow to set a configurable chunk size). If not set, then they could all use a different chunk size as different format may have different size expectation.

I should make sure every subclass of `Converter` use this setting (even if they use a library to ease the implementation, they must use libraries that allow to set a configurable chunk size). If not set, then they could all use a different chunk size as different format may have different size expectation.

linarphy added a new dependency

2026-06-09 16:28:56 +00:00

#12 Add export and import method for graph

linarphy self-assigned this

2026-06-09 16:30:10 +00:00

linarphy added this to the Roadmap to 1.1.0 project

2026-06-09 16:30:15 +00:00