You can make the Address Line 2 field required by going to Customize > WooCommerce > Checkout and setting the Address line 2 field to required:
To make only the shipping address line 2 field required, you can use this snippet:
The same pattern can by used for the billing address – just replace the snippet references to ‘shipping’ with ‘billing’.