beat.backend.python issueshttps://gitlab.idiap.ch/beat/beat.backend.python/-/issues2020-06-08T14:00:56Zhttps://gitlab.idiap.ch/beat/beat.backend.python/-/issues/33Add multi-processing support to RemoteDataSource2020-06-08T14:00:56ZSamuel GAISTAdd multi-processing support to RemoteDataSourceFollowing #32
The current implementation of the processing block to database container uses a one to one connection through ZMQ exclusive pair.
In order to also allow people to use multiprocessing with a dataset as input to a block, th...Following #32
The current implementation of the processing block to database container uses a one to one connection through ZMQ exclusive pair.
In order to also allow people to use multiprocessing with a dataset as input to a block, this much be changed to something different. Essentially multiple clients to one server in request/response mode.https://gitlab.idiap.ch/beat/beat.backend.python/-/issues/31Support bytes python type2020-05-27T16:23:17ZSamuel GAISTSupport bytes python typeCurrently there's no way to send bytes directly from one algorithm to another.
The current "workaround" is to build the bytes output then use base64 encoding, use a string based type to write the data to the output and on the other end ...Currently there's no way to send bytes directly from one algorithm to another.
The current "workaround" is to build the bytes output then use base64 encoding, use a string based type to write the data to the output and on the other end load it back from string.
Example of working code:
Output:
```python
obj = dict(field1=data.value*2, field2=data.value)
dumped = pickle.dumps(obj) # binary
encoded = base64.b64encode(dumped) # base64 bytes
string = encoded.decode("ascii") # "stringified"
outputs["out_data"].write({"value": string})
```
**WARNING** Do **not** call str(encoded), it won't be a "real string":
```python
example = b"whatever"
example_str = str(example)
example_str
# output is "b'whatever'"
```
Input:
```python
in_data = inputs["in_data"].data # string data
decoded = base64.b64decode(in_data.value) # decode to bytes
obj = pickle.loads(decoded) # obj is a dict and can be read
```
This issue tracks the implementation of the support for the python bytes type.