Immune checkpoint inhibitor (ICI) therapies can lead to autoimmune toxicities known as immune-related adverse events (irAEs). In a study involving 77 cancer patients receiving ICI therapy, we explored the use of a new metric, the T-cell tolerant fraction, calculated from baseline T-cell receptor beta (TRB) sequences, to predict the occurrence of significant irAEs. We observed that patients with a lower tolerant fraction experienced more pronounced irAEs. Specifically, this relationship was statistically significant for those receiving CTLA4 therapy, with some indications in other ICI categories. Moreover, the T-cell tolerant fraction was more reliable as a predictor (AUC of 0.79) than measures of TRB clonality or diversity. The findings suggest that the T-cell tolerant fraction at the baseline could be a potential indicator of significant irAEs in ICI-treated patients.