Context: Serum alpha-fetoprotein (AFP) has been shown to be valuable in tumor staging and predicting survival outcomes. In this investigation, we conducted a retrospective cohort analysis and a meta-analysis to assess the predictive significance of initial AFP levels in patients with hepatocellular carcinoma (HCC) who underwent treatment with immune checkpoint inhibitors (ICIs). Methods: We searched databases from inception until 14 July 2024 to identify cohort studies involving ICI treatments in HCC patients with baseline AFP data. We also retrospectively analyzed patients with HCC treated with ICIs to assess the therapeutic effect in the high AFP (AFP >= 400 ng/mL) group and the low AFP (AFP < 400 ng/mL) group in terms of overall survival (OS), progression-free survival (PFS), objective response rate (ORR), and disease control rate (DCR). Results: In the meta-analysis, a total of 34 studies, comprising 8,799 patients, were included, while the retrospective cohort study encompassed 55 patients. In the meta-analysis, the summarized hazard ratios (HRs) of AFP >= 400 ng/mL versus AFP < 400 ng/mL for ICI therapy indicated that the high AFP group had a poorer outcome compared to the low AFP group, with a pooled HR for OS of 1.69 (95% CI: 1.57 - 1.82, P < 0.001) and a pooled HR for PFS of 1.47 (95% CI: 1.33 - 1.63, P < 0.001). In the retrospective cohort study, higher AFP levels were associated with a lower DCR for ICIs, with a DCR of 42.9% in the high AFP group and 77.8% in the low AFP group (P = 0.008). Cox model analysis showed that higher serum AFP was an independent predictor for shorter OS (HR 3.584, 95% CI: 1.466 - 8.762, P = 0.005). The toxicity analysis also displayed a strong association between high AFP and the occurrence of immune-related adverse events (irAEs) (P = 0.008). Conclusions: Higher serum AFP is associated with poorer efficacy of ICI treatment in HCC patients.