Thank you all for the advice! We will definitely take it to heart.
I should have said we are NOT walking him outside in the general world, just walking with him around our dog-free backyard. Our "correction" for growling has been backing away and not engaging with the scary situation quite yet, so it seems like we need to continue to allow him to set the pace with any interactions with people we have in our home.