The overall survival (OS) remains unsatisfactory in patients with esophageal squamous cell carcinoma (ESCC) after extended esophagectomy with two-field lymphadenectomy. Therefore, this retrospective study aimed to identify the risk factors that contribute to the low survival of patients with pT1-3N0M0 ESCC. Patients with pT1-3N0M0 ESCC who only underwent R0 esophagectomy with two-field lymphadenectomy in our department from January 2008 to December 2012 were retrospectively enrolled in this study and medical records were reviewed. Postoperative OS, disease-free survival (DFS), recurrence-free survival (RFS), and locoregional recurrence-free survival (LRFS) were analyzed sequentially. This study recruited a total of 488 patients, whose follow-up visits were completed at the end of December 2019. The five-year OS, DFS, RFS and LRFS rates were 62.1, 53.1, 58.3 and 65.6%, respectively. Multivariate Cox analysis identified patient age, site of the lesion, small mediastinal lymph nodes in CT imaging (SLNs in CT), dissected lymph nodes (LNs), and stage of esophageal malignancy as independent risk factors for OS of the patients. Of these factors, the site of the lesion, SLNs in CT and stage of the cancer were determined to be independent factors for DFS, RFS and LRFS. Based on all five factors, the recursive partitioning analysis (RPA) score system was developed to stratify the patients into low-, medium- and high-risk groups, which were found to possess significantly different rates of OS, DFS, RFS and LRFS (p < 0.001). Several factors were associated with the survival of patients with pT1-3 N0M0 ESCC who underwent extended esophagectomy with two-field lymphadenectomy. These factors contributed to the RPA scoring system, which could stratify the risk of postoperative survival and may expedite the initiation of postoperative adjuvant therapy.