Niallo I can understand it's frustrating to not get the site speed scores that you are hoping for. Ezoic is a data company and we recommend making decisions for your site based on data. The best way to help your site increase their core web vital scores is to fully set up Leap as this has been specially designed to help you to understand why you are getting the core web vital scores that you are, help you to resolve these with the optimization settings.
You mentioned that your main concern is ttfb this article explains what impacts time to first byte https://en.wikipedia.org/wiki/Time_to_first_byte and this article explains why this metric isn't the best one to monitor https://www.ezoic.com/ttfb-shouldnt-matter-test-this-instead-for-pagespeed/
Google launched the core web vitals last year and the three main areas that they are monitoring are the Loading: Largest Contentful Paint (LCP) Interactivity: First Input Delay (FID) and Visual Stability: Cumulative Layout Shift (CLS) This article explains more about this in full detail: https://www.ezoic.com/how-to-optimize-core-web-vitals-in-google-search-console/
To help you to improve your core web vitals we would recommend setting up Leap and then waiting two weeks to see what impact this has on your core web vital scores. If you have concerns after this time please do reach out to us and we can then go from there. thank you