Yes, a thief could certainly take the whole cabinet.   Again, that wasn't the intent here.  If it were me and I wanted to prevent that, I'd have bought a decent safe and anchored the thing in the basement into concrete.  This is intended for scenarios where someone - that likely you know (e.g., a child or grandchild) - comes upon this cabinet, finds it locked, and that indicates that they shouldn't go in.   This is similar (albeit more substantial) to a privacy lock on a bathroom.

?