Tip – fit_generator in keras – how to parallelise correctly

Seems like many got confused with it, at least when they relying on the documentation. There are quite a lot of github issues including #1638. With a deep understanding of Python it might be trivial. For me, it wasn’t.

There are three input arguments that are related to this issue. (Documentation)


max_queue_size=10, workers=1, use_multiprocessing=False

  • max_queue_size : It specifies how many batches it’s going to prepare in the queue. It doesn’t mean you’ll have multiple generator instances. See the example below.

Unlike a comment I saw in some keras/issue, it doesn’t mean the training begins only after the queue is filled. Making the yield super-slow shows this.


  • workers , use_multiprocessing: with the naive input loader it fails. with the naive input loader + use_multiprocessing=True, it works with many generator instances. But you should make it sure if this is what you want. If you see the result, the indices will help you understand what’s going on.

Here, I knew generators are non-picklable, so actually don’t understand why it still works with multiprocessing. But what’s happening is obvious with the printed logs. People report some thread problems here in keras/issue.

  • With a thread-safe generator?

With a thread-safe implementation, multiple workers have no problem with both use_multiprocessing=True/False. Multiple generators are instantiated only when use_multiprocessing=True.


My conclusion: make it thread-safe and perhaps turn on the multiprocessing option if you think you’d need it to speed up.

6 thoughts on “Tip – fit_generator in keras – how to parallelise correctly

  1. Looks to me the threadsafe_iter is just locking one iteration until completion, and then releases it. This kinda defeats the purpose of having multiple threads, does it not?


  2. Hi! thanks for this examples. I have a question: so, when we set use_multiprocessing=True and more than one worker, are we feeding duplicated data to the network?

    Liked by 1 person

  3. You can use keras.utils.Sequence class, which takes care of multiprocessing concurrency for you:

    From the webpage:
    “Sequence are a safer way to do multiprocessing. This structure guarantees that the network will only train once on each sample per epoch which is not the case with generators.”


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s