Grokking Deep Reinforcement Learning

It is a good overview of deep reinforcement learning. The code and equations on the kindle version are difficult to read. I would suggest this to someone very new to the field, especially with limited coding experience. The one area I wish it had hit on better is the development of new environments, this is especially difficult area in research for high dimensional problems.

The book covers both tabular RL and Deep RL. The author has tried to explain concepts in a very intuitive way by avoiding going deep into maths. However, all the necessary maths has been discussed completely and intuitively. One great point of this book is that it has explanations of many recent major deep RL algorithms.

Overall, this book is remarkable. Very clear, easy to ready. Just a gem.The standard text on reinforcement learning is Sutton and Barto. However, I found myself wanting more code. This book delivers *both* more accessible explanations and more code. It uses a very similar notation to Sutton and Barto and uses a similar ordering of topics -- at least for the tabular material. So I've found reading those books together to be useful.Except, it needs a new revision. There are several irritating typos along the ways including a few sentences that seem to be just word jumbles. As others have pointed out, the code examples in later chapters rely on out of data libraries. Some samples no longer run.If this book were up to date, had a few tweaks, and the code was fixed, I would easily give it 5 stars. In it's current state it's four.Deep Reinforcement Learning Hands-On by Lapan, which I do not recommend, just got a third edition. I think it's time this one got an update.

Sutton's book is the bible of RL. But this book is the best introduction to RL.- Fantastic explanations of complex RL concepts.- Perfect balance of mixing math and Python code.- Answers many of the questions that other books don't, all the gotchas of RL.What Miguel has done with this book is amazing.

This book is an incredible compliment to Sutton. Sutton is great for theory, but Grokking Deep RL will bridge the gap to writing real code that solves real world problems.

I had trouble picturing how the Grokking series would be able to cover such a complex topic as deep reinforcement learning. I came to this book with prior exposure to Reinforcement Learning and a short prior exposure to Deep Reinforcement Learning. So while I can’t quite give the perspective of a complete novice, the book did strike me as being very well organized and explained. What really sets this book apart is the annotation of both math formulas and Python code.One helpful feature of the book was the “Refresh My Memory” sections. In them, the author summarizes relevant concepts presented in earlier chapters just in time to apply them to the new material of the current chapter. This saves the reader from flipping backward to search for the prior references. I found these sections to be completely sufficient to keep moving forward and absorb the new material. I also like that the author explains not just how the algorithms work, but why they work, and presents plots that compare the performance of different algorithms.The code and Docker instructions on the author’s github repo were very easy to follow (I already had Docker installed). Since I had previous experience with RL, I have not run all of the code, but I did run the code for the more advanced algorithms in chapters 9, 10, and 12. The code itself is well written and clean. The notebooks in early chapters have plenty of comments to help understand what is happening.I caught almost no typos, which surprised me for a first edition of a sizable book. The writing style is conversational, which helps make a technical subject much less dry.In summary, if someone were to ask me to recommend one book to take them from a novice to a capable deep reinforcement learning practitioner, I would recommend this one. I would recommend that a reader actively engage with the provided code. Add doc strings and comments to the code as you read it. Then try to rewrite the algorithms yourself from just the math and explanations, or translate the PyTorch portions to TensorFlow to really test your understanding.

This book will take you from the very basics of Markov Decision Processes (which form part of the foundation for RL) to state of the art Deep Learning Methods to approximate more difficult environments. The analogies he gives for the techniques are also gold because it gives you an idea of the math present (which is also explained in great depth) and are great if you teach a class and want to drive a point home to a lost student. The code is also explained line by line and he gives you the code to try the results that he derives on your own and experiment with them. There's no reason not to get this book if you want a strong foundation in RL. I'd suggest it as a starting book over Sutton and Barto (which is a great resource on it's own. But this will give you more code examples and will help lay a better foundation for you to appreciate it more).