Generally, Critical CSS sets inline in and defer all non-critical styles that are loaded asynchronously. So we can significantly improve the time to first render of site.

The ‘critical’ CSS file should be located in app/design/frontend/themevendor/themename/web/css/critical.css

After setting critical CSS, you need to enable it from the backend side. steps are Stores > Settings > Configuration > ADVANCED > Developer tab. Make sure this option does not show if production mode is enabled.

In Magento default, the luma theme has critical.css so you can take reference from there, and here is the official doc in case if you wish to read further.