Are you using the Asset Store version or the eval version in this project?
In any case, you can manually add USE_PHYSICS2D by selecting menu item Edit > Project Settings. Then add it to Player -> Other Settings > Scripting Define Symbols and click Accept. Or select menu item Tools > Pixel Crushers > Common > Misc > Enable Physics2D Support. That will add the scripting define symbol for you, too.
I've noticed this with the academic version and the evaluation version (while I personally own the asset pack, I am a college teacher that you have shared the academic version with in the past and I have my students use it for a class). I did use the steps you shared as a workaround, but I think that since the tutorial videos use 2D, it might be nice to enable this checkbox.
Or maybe just adding a note about Tools > Pixel Crushers > Common > Misc > Enable Physics2D Support to the tutorial video at the timestamp where you mention checking the box would be the simplest solution. Just suggestions, thanks again for your continued support!
I'll double check that the Welcome Window shows the USE_PHYSICS2D checkbox as ticked in the academic and evaluation versions. The DLLs in those versions are compiled with the USE_PHYSICS2D scripting define symbol, so it might just be a UI issue.