Sparse data structures — pandas 1.4.0 documentation
pandas.pydata.org › stable › user_guideAs you can see, the density (% of values that have not been “compressed”) is extremely low. This sparse object takes up much less memory on disk (pickled) and in the Python interpreter. In [11]: 'dense : {:0.2f} bytes'.format(df.memory_usage().sum() / 1e3) Out [11]: 'dense : 320.13 bytes' In [12]: 'sparse: {:0.2f} bytes'.format(sdf.memory_usage().sum() / 1e3) Out [12]: 'sparse: 0.22 bytes'.