DigiNews

Summary

The post argues memory efficiency matters as RAM becomes scarcer on consumer devices. It compares a Python script that reads a UTF-8 text file, tokenizes on whitespace, counts word frequencies, and prints results with a peak memory of about 1.3 MB, to a fully native C++ version built with Pystd that does the same in roughly 20 lines of core code. The native approach uses mmap to map the file, validates UTF-8, creates a view into text, lazily splits into words, and stores counts in a hash table that uses string views rather than allocating new strings. Memory usage is around 100 KB for the native version, about 7.7 percent of the Python version. The author notes that Python has a startup cost and that the comparison can be made even harsher by removing exception support from the C++ build, which could reduce total memory to roughly 21 KB, potentially a 98.4 percent reduction. The piece emphasizes that avoiding dynamic memory and string allocations through techniques like string views and memory mapped input yields large memory savings, and it presents a tradeoff between Python's ease of use and native code's lean footprint.