Couple of questions:
Unfortunatelly ARM32 and ARM64 are not Tier 1 architectures (only i386 and amd64 are) so there are not updates with freebsd-update. This will change with introduction of FreeBSD 13.0-RELEASE … at the beginning of 2021 I think.
It’s understandable they don’t provide binary updates, but I was surprised I couldn’t just make a buildworld from the pi itself following this. I have to admit I haven’t looked too much in details, I will try again later.
The make buildworld process sometimes fails even on *-RELEASE versions.
Submit a bug here - freebsd.org/send-pr.html - or describe your problem on the FreeBSD Forums - forums.freebsd.org - you will get lots of help there.