The coinhibitory molecules, B7-H3 and B7-H4, have shown negative regulation in T cell activation and tumor-associated macrophage (TAM) polarization in tumor-specific immunity. Here, we investigated the expression of B7-H3 and B7-H4 in human and murine esophageal squamous cell carcinoma (ESCC) tissues to define their clinical significance and mechanism in a tumor microenvironment. In the present study, B7-H3 and B7-H4 were expressed in 90.6 and 92.7 % samples, respectively. High B7-H3 and B7-H4 expression was associated with advanced TNM stage and lymph node metastasis (p < 0.05, respectively). Patients with both B7-H3 and B7-H4 high-expressed tumors had the poorest prognosis (26.7 months), whereas those with both low-expressed tumors had the best survival (56.7 months). B7-H3 and B7-H4 expression were inclined to be positively related to the infiltration intensity of Treg cells and TAMs (p < 0.05, respectively), and B7-H3 expression is negatively associated with the intensity of CD8(+) T cells (p < 0.05). In 4-nitroquinoline 1-oxide (4-NQO)-induced murine models, high B7-H3 expression could only be detected at carcinoma stage, but abnormal B7-H4 expression appeared a little earlier at dysplasia stage. In vitro studies revealed that knockdown of B7-H3 on tumor cells suppressed ESCC cell migration and invasion, while knockdown of B7-H4 could inhibit ESCC cell growth. Overall, B7-H3 and B7-H4 are involved in ESCC progression and development and their coexpression could be valuable prognostic indicators. Interference of these negative regulatory molecules might be a new strategy for treating ESCC.