Repository navigation
Python: high performance backend #8
Description
Activity
As far as I understand, the python implementation is fully functional.
I think we should make this issue more precise. E.g. by specifying what performance problems you see right now. Some benchmark numbers could also help. This would enable us either to split this issue or introduce a precise check-list what needs to be done.
I'm curious, do you already have something that we could commit to produce performance figures? i.e. compare C++ implementation vs the Python implementation with different Python runtimes (CPython, PyPy, ...)
More Efficient Python Implementation
Current
flatdata-pyimplementation is pure python. So far we have used it only for processing smaller datasets and for inspection/debugging. It was noticed that on large datasets it performs quite slowly. It would be useful to have an implementation with performance not too far from C++ one. In order to achieve that, we could do following:dask).