Understanding Memory Constraints in Image Classification
Image classification usually involves training algorithms to identify given objects in an image. It sounds like a reasonably straightforward matter but one is bound to fall into the many traps it holds for newcomers.
One such elaborate trap is dealing with memory constraints. When developing a model, it is necessary to have good training data in order for it to work optimally. It is easy to think that once you have the data then everything should align perfectly but then as you slowly progress, it becomes evident that you may have underestimated the severity of the task. Once you get to the part where you are supposed to train your model you receive a warning that “There is not enough memory for the task.” What do you do?
The above scenario arises if you’re working with numerous images, have inadequate memory and you’re manually converting your images to Numpy arrays in order to feed them into the model.
How Numpy Array Conversion Works
When working with images, one of the first operations that you perform is the conversion of image data to arrays. Normally the step looks like this:
Image — — — — > Array — — — — > Numpy Array
You convert the image to an array, normally using a library such as OpenCV and then you specify the X-variable(features) and the Y-variable(labels). After making this specification, you will convert the data into Numpy arrays using the Numpy library.
The Numpy Array is an efficient data structure that allows versatility when performing mathematical calculations. Unfortunately, efficiency does not mean perfection in all situations. Converting many images into a large Numpy array could lead to multiple Gigabytes of information being formed.
If you are short of memory, then performing tasks such as normalization of the Numpy array will lead to the entirety of your memory being occupied by it. The operation will occur in the memory and if the size of the Numpy array exceeds the size of the memory then your computer will halt all operations.
There are a few workarounds to being memory constrained, despite the valid concerns mentioned above. Intelligent minds have thought through the situation and have cleverly crafted ways to avoid this problem.
“Just Eat Cake…”
You don’t have enough memory? Why don’t you just buy more?
The most straightforward solution is to just get more memory. Unfortunately, if this were the easier option for everybody then I wouldn’t be writing this article.
It is possible that we will always run into memory shortages depending on how our work scales. So buying a few more Gigabytes of memory might be useless if you end up working with exponentially more data than it could manage. This is why learning how to manage your memory is a key lesson when dealing with image classification.
One of the greatest assurances we have when working with such memory constraints is that the major libraries that we have in use have this thought out and sorted. Below, we will look at NumPy and other solutions to this problem.
Team wide collaboration is a core of good MLOps strategy. Learn what else makes effective ML teams work with our free ebook.
Numpy memmap()
After creating a Numpy array, you can store its binary as a Numpy file. Once you do this, there is no need to keep on repeating the process of converting images to Numpy arrays as you will already have all the information stored in your local drive as desired.
Once the Numpy binary is in the storage, you do not have to load all the information into memory when trying to train a model. This is where Numpy’s “memmap()” comes in.
According to Numpy’s documentation, its job is to “create a memory-map to an array stored in a binary file on disk.” It allows you to efficiently access small sections of a binary. This reduces the amount of memory you need to be using at a time. The system allows an individual to perform quick batch creation without having to load everything into memory.
Another additional benefit that arises when using memmap() is that it allows you to perform many of the same operations that you can perform on Numpy arrays.
Ignacio Oguiza’s notebook on the topic gives a deeper dive into the topic as he gives multiple examples of how to effectively use Numpy’s memmap(). Now we can move to the next task of trying to avoid memory problems.
Data Generators and DataLoaders
Dataloaders and Data Generators are provided by Pytorch and Keras APIs respectively. A programmer can tactfully use them to ensure that they code all their functionality such as transformations in them without overwhelming the computer’s memory.
Two important functionalities that they have include balancing the number of labels in datasets and allowing small batches of data to be fed into the model at a time. It is easy to set the batch size at a time according to your requirements and according to your memory capacity.
The preference for using either boils down to the needs of the programmer. Pytorch may feel more natural to those who are used to using Numpy while Keras may feel easier to those who have been using it over time.
Conclusion
Now we have a good understanding of where our memory goes when we are doing our memory-intensive image processing applications. We know that it doesn't have to be that intensive as there are tools built to cater for those that do not have adequate memory.
Understanding the use of memmaps, dataloaders and data generators is necessary for memory-constrained applications. There are many advantages and a few disadvantages to all these. The underlying importance that you need to consider is the fact that you may not have enough memory for a task.
Editor’s Note: Heartbeat is a contributor-driven online publication and community dedicated to providing premier educational resources for data science, machine learning, and deep learning practitioners. We’re committed to supporting and inspiring developers and engineers from all walks of life.
Editorially independent, Heartbeat is sponsored and published by Comet, an MLOps platform that enables data scientists & ML teams to track, compare, explain, & optimize their experiments. We pay our contributors, and we don’t sell ads.
If you’d like to contribute, head on over to our call for contributors. You can also sign up to receive our weekly newsletter (Deep Learning Weekly), check out the Comet blog, join us on Slack, and follow Comet on Twitter and LinkedIn for resources, events, and much more that will help you build better ML models, faster.
