Can Artificial Intelligence Be Safe?

摘要

Many of the recommended practices for generating safety related software involve using computer tools that themselves qualify for the attribute "artificial intelligence", although this is not always evident. A conversion from free text to a formal notation is admittedly done by highly qualified humans, but most of the rest is done by machines. Depending on the outcome of the intermediate computer programs that are applied, either a modification of the input and a subsequent re-run of the corresponding program will be required, or the next program can be started immediately. Since the programs all provide electronic output, it would be a simple task to automate the entire process, for example by using suitable scripts. In addition, the formal proof for railway interlocking systems relies on sophisticated computer programs to perform complex logic tasks. So effectively, "logical artificial intelligence" is already being used in safety related systems, or to answer the question in the title of this paper: Yes, it already is! That being said, the next step must be to develop methods for assessing the safety of artificially intelligent systems. Certainly some of the technologies will be able to be regarded as proven in use, but for more sophisticated technologies it will take a long time to reach the amount of experience that is necessary to qualify. For safety related software, the standards such as recognise that a quantitative demonstration of safety is not possible. Therefore, a qualitative approach is used. By demonstrating that well structured and controlled processes have been applied, it is possible to make a decision about the quality of the software and infer its suitability for a safety related application. Currently, the standards do not regard artificial intelligence as a well structured and controllable process, so they reject it. This is somewhat contradictory, because an artificially intelligent system is a programmed system, so if the acceptable methods and tools are applied when the artificially intelligent system is created, it should be possible to qualify it for a safety related application.