I finally spent the time to figure this out. It’s easier than I thought:
find . -type d -exec test -e {}/.git \; -prune -o -type d ! -path . -print