When loading a large set of logs and memory is insufficient, what is the best approach?

• A. Use Internal Storage For Processing
• B. Analyze The Logs For Errors
• C. Perform Chunking On The Data
• D. Increase The Memory Size Of The System

Answer: (C)

When data is too large to fit in memory, processing it in chunks keeps memory usage bounded. By dividing the large log set into smaller portions that fit into RAM, you can load a chunk, analyze or transform it, then move on to the next chunk. This streaming or chunking approach lets you handle massive datasets without needing to load everything at once, making the process scalable and resilient to memory limits. Relying on the entire dataset in memory isn’t feasible here, and while increasing memory could help, it’s not always practical or cost-effective. Analyzing for errors is a task you perform on the data, but it doesn’t address the memory constraint by itself.