The inventors of the present disclosure have identified that most standard machine learning approaches fail in solving such problems due to the very low rates of out-of-spec wafers (typically 0.3 - 1.5) and because of how KPIs are designed (each KPI is particularly designed to address a specific problem). In addition, it is often assumed that the KPIs are sufficient to make such detections whereas in tact there might be cases where there arc no KPIs that can explain why a wafer is out-of-spec. In addition, many KPIs are correlated with each other. All the above are strong indications that the trained classifier/regressor is likely to overfit the data regardless of the effort that is spend in optimizing it. The inventors of the present disclosure have identified that another approach to identify the KPIs that could potentially lead to the detection of out-of-spec wafers would be to perform an exhaustive and brute force search on all available KPIs. During this search, for every KPI all the individual thresholds together with different thresholds for KPI-combinations could be evaluated. This way of detecting out-of-spec wafers is suboptimal in providing any diagnostic solutions. Moreover, the threshold is determined on the whole population of the wafers that are exposed over a large period of time, and lhat limits the detection performance: Interaction between different subsystem failures are neglected in the single KPI approach, and that this is necessary to capture unknown and complicated failure modes. In case KPIs are fine-grained, and not mature enough to capture a subsystem failure, then they cannot be used for wafer failure detection. Aspects of the present disclosure are directed at to automate the identification of Failure Modes (FMs), to enhance the detection capacity of out-of-spec wafers and to provide diagnostics. According to one aspect of the present disclosure there is provided a computer implemented method of determining a classification model comprising KPI thresholds for classifying product units subject to a process performed hy an apparatus, the method comprising: receiving KPI data obtained as a result of the plurality of product units being subject to the process, the KPI data associated with a plurality of components of the apparatus and comprising data associated with a plurality of KPIs; clustering the KPI data to identify at least one cluster; analyzing the at least one cluster to identify a plurality of failure modes associated with the apparatus, wherein said analyzing comprises, for one or more of the at least one cluster, identifying a plurality of sub-groups of KPI data relating to a failure of a product unit, each of the plurality of subgroups of KPI data associated with a failure mode of the plurality of failure modes; and determining the classification model by assigning, for each identified failure mode, a threshold to each KPI associated with the failure mode. The method may comprises projecting the KPI data to a lower dimensional space prior to performing the clustering. The method may comprise projecting the KPT data to a 2-dimensional space. The identifying a plurality of sub-groups of KPI data of a cluster may comprise determining lhat a first distance between KPI data points in the cluster that are associated with a failure exceeds a second distance associated with all KPI data points in a largest cluster of the at least one cluster. The first distance may correspond to a first principal component identified hy performing principal component analysis on the KPI data points in the clusier that are associated with a failure, and the second distance is identified by performing principal component analysis on the KPI data points in the largest cluster. The second distance may be a predetermined percentage of a length of a first principal component identified by performing the principal component analysis on the KPI data points in the largest cluster. The identifying a plurality of sub-grou
展开▼
