Recent experiments reported an antisymmetric planar Hall effect, where the Hall current is odd in the in plane magnetic field and scales linearly with both electric and magnetic fields applied. Existing theories rely exclusively on a spin origin, which requires spin-orbit coupling to take effect. Here, we develop a general theory for the intrinsic planar Hall effect (IPHE), highlighting a previously unknown orbital mechanism and connecting it to a band geometric quantity - the anomalous orbital polarizability (AOP). Importantly, the orbital mechanism does not request spin-orbit coupling, so sizable IPHE can occur and is dominated by an orbital contribution in systems with weak spin-orbit coupling. Combined with first-principles calculations, we demonstrate our theory with quantitative evaluation for bulk materials TaSb2, NbAs2, and SrAs3. We further show that AOP and its associated orbital IPHE can be greatly enhanced at topological band crossings, offering a new way to probe topological materials.