33 ES architecture and Knowledge Engineering

Bhushan Trivedi

Introduction

We have Introduced ES in the previous module. As number of ES in the world is very large and they are applied in diversified areas of interest, it is hard to give a general architecture of ES. However, we will try to narrate general features and components of architecture which are used by most ES solutions in this module. We will also discuss the roles of each component in brief. Additionally we will also try to see how ES gains the knowledge and process it in brief. At the end, we will see how ES can be categorized based on their level of expertise.

ES Architecture

There are a few attempts to have a generalized ES structure and even build an ES Shell which allows knowledge storage and inference, there is little interest in generating standalone ES architecture in recent past. It is primarily due to the reason that we have ES components embedded in general systems rather than having a standalone systems.

Anyway, the study of the architecture will enable us to understand the components needed for ES and thus it helps us understand the design needs for problems needing ES solution. Even when it is to be embedded in general systems, all or most of the components are anyway needed.

The figure 33.1 indicates general architectural components needed for an ES.

Figure 33.1 General ES architecture

Let us look at each component in detail

Query processor and client modelling

This is the module user interacts with. All commercial systems have verities of the users and ES is not an exception. Generally, the systems have following types of users.

Administrator: – this is a user with ability to maintain user accounts, rights and related things. We will not discuss about him further as such a user also exists on other commercial systems. He is responsible for creating all other types of users and defining their roles and rights.

Normal User: – this is the most common type of user for ES. For example he is an admin whose job is to look after the complete network in the case of Security ES being developed. He is a patient for a medical ES. He is a teacher or student for a tutoring ES. The acceptability of the ES depends highly on this user’s response to the system. Other users may or may not exist but this user is always there for any ES.

Expert: – This is a user roughly divvied in two categories.

First is an expert who is trusted for providing knowledge to the ES. He will add facts and rules that we have seen examples of in the previous module. Sometimes even machine learning is provided for the system to learn. For example it is almost impossible for a human doctor to provide all information related to human body, complete information about all diseases, and complete information about all possible drugs and so on. There has to be a mechanism for the system to learn using some automated means. The expert user will have to bear the responsibility of introducing that process if he is not doing it manually. Another option is to use a manual interface and take help of another user called knowledge engineer who can introduce knowledge in the system after gleaning it from the expert. Thus acting as a mediator between the expert system and the expert.

Second type of expert user is a verifier or quality analyst, whose job is to provide different cases to the ES and see its responses. For example an expert for a medical domain, he will have to design test cases which can check for all possible diseases the ES is designed to handle plus many other normal cases to see how effective the ES isfor a medical domain. In short, this expert acts like an evaluator or tester for the expert system. Usually multiple experts are provided with this duty.

In fact both types of expert users have to act in sync, the knowledge providers must be given proper feedback by the quality analyst and knowledge provider must take those feedback to improve the system.

Though the roles of both types of experts look similar to the roles of developer and tester in a commercial system, there are a few differences. First is the introduction of heuristics which is a biggest difference in ES. Another is the ability to explain its reasoning which requires the knowledge to suit explanation for a typical user. Yet another is the possibility of the system to come back with a wrong or partially incorrect answer and dealing with them. Last two points will be elaborated during discussion that follows.

Learner: – unlike other systems, an Expert System deals with a typical domain which expects a specific level of understanding. When the ES is deployed in an organization, some users may be well versed with the domain and can straight away start working with the system as an expert for either providing knowledge or testing the ES. Unfortunately not all users can straight away start working as an expert. For example an Intelligent Tutoring system is designed to generate practical test papers and their evaluation in an automated form1. Not every teacher will be able to start using the system. “1 One of the author’s Ph. D. Student is working on this problem” Those who are unfamiliar to terms related to pedagogy needs explanation form many topics like types of learners and how one needs to mould the pedagogy for class containing a typical mix of them, bloom’s taxonomy, questions to address different bloom levels, how to assess a bloom level associated with a particular question, how to rate the question and so many more things. This type of user is interested in learning about the domain from the expert system. In fact most expert systems have an important sub-goal of gleaning knowledge from experts and distribute it among learners to improve overall functionality of the organization2.

In conventional client server architecture, this module invariably resides on the client. However, processing their queries might happen on the server. Our meaning of processing the query is different than conventional SQL based or NoSQL based queries. It is expected that the ES provides more human like interface and thus the natural language interface or voice interface or handwritten character based interface, or hand drown figure interface needs processing for learning what is being conveyed. That part of processing, so far, is not part of the commercial systems though they have started being used in authentication processing3.

Interface

When we are dealing with multiple users as described above, it is important that our interface is flexible enough to deal with each one of them. Interestingly, ES requires more elaborate interface for two reasons.

First the ES has to provide explanation which suits the TYPE of user. For example a  program debugger might respond like “The memory allocation failed due to stack being full” to a programmer but if such a response if provided to a normal user, he won’t understand it. The response might be tone down to “There is a memory problem, please call the administrator”. A medical diagnosis system might respond back to a referrer “the test case confirms encephalitis”. A normal user might be responded back like “it is a disease which is similar to a TB for the brain”. An IDS might respond back with “A distributed denial of service attack” to an expert but for a learner it might respond back as “the attack is being directed from many other computer systems from the attacker”. Unlike commercial systems, Expert Domains has to deal with jargons. Many words used by the experts are not easy for others to understand. While providing response or explanation to experts those words are possible to be used but otherwise their meaning and sometimes analogy (TB of brain for encephalitis) is to be used.

Second, the ES deals with variety of users and thus the interface needs to be tuned for them like the case 1. The items displayed on the page for example, should be only which the user understands or needs to look at. Also, the user might require additional information as he is not an expert and thus related issues might be addressed.

The interface provides two way communication and thus the input from the users to the system is also an important part. It is a process of accepting information from the user and convert and store it into knowledgebase and take it from knowledgebase and provide it to user in a form that he can understand.

Many ES are designed to deal with typical interface for typical requirements. For example, when the ES is dealing with disabled people, the requirement for the interfaces is quite weird sometimes; for example one system takes their input from eye movements of the user and another from some body part movement exist. A branch of computer science called HCI or Human Computer Interface is designed to deal with such complex cases. Anyway, we will confine our discussion to normal interface.

“2 In fact, this knowledge forms the basis of knowledge management, the term used for managing organizational knowledge.”

“3  Mobile phones and laptops using face recognition or thumb recognition are examples.”

The best interface possible is the natural language. User may speak and type any natural language statement to get his work done. For example user might input “I am feeling weakness from last two days and having slight fever” for a medical expert system to respond back. Unfortunately it is really very hard to understand an arbitrary natural language statement. There are many attempts done in that direction and there is some success but full-fledged solution is not yet possible. It is possible to provide explanation in natural language as the system might contain information in form of natural language statements and provide as the response to the query from the user. The problem here is that it is sometimes misdirected and provide response which does not match the query. This is being researched and there is some success in that area as well but we still await a full-fledged system with a capability of responding a query.

The other possible alternatives are GUIs (Graphical User Interface), APIs (application programing interface) or PLIs or CLIs (program level or call level interfaces), Text and Voice Commands, or menu driven interfaces. The harder part is initiating the process and navigation in the right direction during responding the query. A human expert can directly jump to problem after a few queries typically designed for that particular user. For example when you go to a doctor, he might look at your body (which is weakened), mosquito bites on the face, know that the area that you live in is prone to malaria, might inquire about malaria without looking for other thigs. How an expert system can generate similar powerful interface is still a mystery. Similarly a security expert might look at the system being slow already, some machines use older OS and also have noted that the database is not patched, most servers are unreachable, conclude that some form of DDOS (Distributed Denial of Service Attack) is being deployed using some vulnerability in the OS or database. Such expertise is possible to achieve only after some experience. If the ES is designed to have that learning component (which we have not shown in the architectural diagram), it is possible it to analyse and deduce such rules itself.

It is also important to note that the interface must be tuned with the type of user. Experts can be given short, precise and quick responses. Learners must be given elaborate answers, normal user must be given answers in the form they can understand. The knowledge base cannot store information in multiple ways, it is important for the system to store the information in the single format and convert them to proper dialog form while dealing with typical type of user. Another aspect of the interface is that it must provide interactive dialogue (like human experts) until the user is convinced about his task being done.

One important type of interface is based on machine learning. The system learns from previous attack scenarios recorded, network logs, RFCs, research papers and other articles. Machine learning is important for the system to learn about facts and normal rules. Machin learning is also applied to learn rules from the expert behaviour. For example doctor’s case records describes the symptoms of the patients, doctor’s comments and prescribed drugs. The machine learning solution may study such logs and improves its learning.

Knowledge storage and maintenance

The knowledgebase is to be constructed and maintained by this component. The knowledgebase contains information in many forms using techniques that we have studied in last few modules. Many implementation use conventional techniques of RDBMS and XML based forms. They also use conventional languages to interact with databases and XML based structures for communicating. In some cases the inference logic is also the integral part of knowledge structure. Anyway the inference part and knowledge storage must interact extensively while solving any problem. The inference logic is shown as a separate module in the diagram to indicate that it is connected but not an integral part usually. Inference, as many methods that we have seen clearly indicate, depends on the way knowledge is represented.

This module is divided in two parts. First is the knowledgebase. This is the database containing all facts and rules about the system. This part determines the power of the system. The knowledge storage part (sometimes denoted as knowledge schema or simply schema) impacts other parts of the architecture.

For example if OWL is used to describe the knowledge user has decided to code the ontology for the domain in XML, the inference depends heavily on the design. For example if we store the ontology about attacks. We may decide a typical hierarchy and place Denial of Service attack under availability related attacks. If the attack is also using database vulnerability to stop database from responding back, it is also related to database vulnerability. If some other attack is also related to application related attack so placed under application attack in ontology. Interestingly, this second attack also has something to do with database vulnerability. Now consider two different cases. In case 1 we want to search for any of these two attacks, application related attacks or availability related attacks. As we have already stored attacks using those labels, it gives us complete information about the attack traversing the given ontology. On the contrary, if somebody wants to get information about database related attacks, it becomes harder as it has to visit multiple ontologies to respond back. Similar problems also exist in conventional databases. They use indexing on multiple fields to solve the problem. Here the problem is that associations are not as straight forward and there are many such associations. Second, it is comparatively easy to decide the queries from customers and it is easier to decide items to be indexed which isn’t that easy in ES case. Usually the solution is to forgo the relational model and opt for designs which are better suited but the problem still remains.

Second part is known as knowledge update or knowledge maintenance. We will soon discuss that. Bottom-line is, knowledge representation is a critical issue in ES.

Knowledge Engineering

Most part of knowledge can be machine learned but not easy to do it for the heuristics. Some knowledge sources are difficult to machine learn. For example video recording of experts about some topic of the knowledge base. The option left is to get the information from expert and such knowledge sources in some other way. If the expert does not provide that information himself to the system, we need an intermediary for the same. When an intermediary is used to provide knowledge, the process is known as knowledge engineering.

The knowledge engineering is not only about taking data from an expert and feed that in the system. In fact it is not a very good idea to take the data as it is. One needs cleansing and normalization operations to be performed before taking data in. The knowledge engineering is about getting such knowledge and placing it into knowledge base usually by intervention of another type of expert; known as knowledge engineer. The knowledge engineer is expert in learning what expert is describing and also expert in entering that information in the system after the required conversion. Other commercial systems does not have this problem. The experts have two problems, though they have many rules of thumb but they neither good at describing them in a form that system takes them in (for example predicate logic form or frames or something similar), neither have they time to learn to do so. An expert is really an expert if he is in huge demand and thus true experts hardly have time.

The knowledge engineer’s first job is to elicit domain knowledge from experts. The subject expert’s knowledge usually is described in the form that their peer can easily understand but not others. The knowledge engineer must be acquainted with the terminology of the domain so he can understand what expert is talking about. The KE (Knowledge Engineer), after acquiring the knowledge from the expert and convert it into the form that can be entered into the system, enter that into knowledgebase. For that KE has to have the knowledge stored in the system, the overall knowledge structure and where each knowledge part is to be entered. For example when an expert talks about a new detection algorithm, the KE must know where to enter the information of that detection algorithm and in which form. Suppose the expert provides the new detection algorithm in Java, and the knowledge structure is in OWL, the java code somehow is to be connected to OWL representation if that Java code refers to those OWL entities. If new types of attacks and their symptoms are provided by the expert in plain English, the KE must convert them to proper naming form (Every attack might have a typical name based on naming convention to keep their names unique), also their place in the hierarchy must be decided based on their attributes (for example if the attack is insider attack or outsider attack, is it related to database or network protocol, if it is a simple attack or a complicated serious targeted attack and so on). It is important for the KE to learn about those characteristics of those attacks and place them properly in the context which correctly defines them4. Sometimes he will also have to decide the weights of a parameter. For example how much weight he provides to system being slow or system files being opened or some failed tries to login as administrator and so on for deciding about a typical attack and deciding what the attacker is up to. He might take help from the expert but the exact description of the rule must be provided by the KE.

In fact it is not as easy as it seems. For this process, the KE must learn about almost everything expert considers fundamental. Following is an example.

If you are comfortable with following computer science topics you might be able to understand the expert’s statement that follows. The network fundamentals, the TCP/IP fundamentals, the OS and Internet fundamentals, types of attacks, terms like authentication, encryption, hash functions, round functions, secure hash functions, and so on. He has to learn acronyms like AES, SSL, PGP, SMIME, IPsec and 3Dsecure and their relevance in the field. He must understand how OS and Networking protocols are related to each other to understand expert’s statement like “It is an IP layer attack, you must enable IPsec at the R1 router, use a Linux command prompt and run the netstate utility to check”. Quite easy! Isn’t it!

Thus the first job that the KE has to do is to learn about the jargon, buzzwords, common phrases etc. and prepare a kind of dictionary himself. He also has to have clear fundamentals of the domain. He must distil the idea from expert’s statements like above and insert that idea in the system.

“4 In some sense, the KE requires the skills of a librarian. A librarian finds where the entry of the new book should go (as per the give classification scheme the library is following) based on the topics the book is covering and place the book in proper book shelf reserved for that class.”

However easy it sounds, this part of the process is hardest for an ES designer. The very communication with the expert, the frequency and the amount of time needed for learning about the domain knowledge is really scarcely available as experts do not have much time5.

In fact, the knowledge acquisition process is still more of an art than science. The heuristic knowledge that is captured through this process, is the true power of the ES being built that is why this is also an extremely critical part of the process.

The inference logic

We have stressed it a few times in previous modules that the ES requires to further its knowledge from many ways to continuously deal with newer and more challenging problems. Take the case of a doctor. He might continuously get new patients describing same set of things in different ways or variations in description of symptoms for similar diseases. Newer diseases appear from nowhere and new innovations in medicine domain and the way patients operated emerge continuously. ES might come out with new rules and also come out with exceptions to older rules. The ES might need to respond to a query using the existing knowledge or something which can be inferred from existing knowledge. ES might need to add new knowledge related to new additions in the domain. ES might also encounter problems which NMRS encountered, default assumptions (for example patient has malaria unless known to the contrary), all dependencies to go out once the assumption becomes wrong (inform the patient to stop taking malaria drugs if the malaria is out of question). The other problem with almost all real world domains is that relatively small number of diseases and symptoms leads to huge number of combinations which ES can hardly be able to manage in real time. The ES operate in high level mode, using rules which are built from primitives but are used directly for diagnosing a disease; exactly like human doctors do. It is interesting to note that when the doctors somehow illustrate their thumb rules about their shorthand methods to diagnose, and the KE asks why that method, doctors have hard time figuring out how they formed that rule. It is because such rules are once generated and then hard coded in their brain, and over the years they forget how they learned those rules.

The module that applies inference logic is known as inference engine. Inference engine uses varieties of approaches we have studied in previous modules including search methods, applying heuristic functions, forward or backward chaining, goal directed reasoning and so on.

Updating Knowledge

The knowledge maintenance module has one more important job. The construction of knowledgebase is the first hurdle that the ES development process crosses. However the second hurdle still remains. Every expert domain expands every day. Newer attacks and newer methods to defend are invented every day for a security domain for example. OS, Browsers, computing paradigms are updating themselves every day, newer applications coming to market, newer protocols arriving, older protocols are refined (latest example is IPv4 is being related by IPv6, SSH is preferred over Telnet and FTP and so on), all in all everything is evolving. Human experts have to read information about new developments or attend workshops or conferences to update themselves. Similarly an ES also has to have some way to update its knowledge base. There are three options. The simplest is to again invite KE to get the updated knowledge in. A more interesting and better option is to invite expert himself to do so. Another option which we have already mentioned earlier is to use some form of machine learning. For example the ES has a module which looks for and ‘read’ articles of interest and try to figure out and add important information inside the knowledge base. Another ability that is not very prevalent is to learn things by itself. Learning is what is being researched heavily as it is still an open research problem. If the ES learns additional information about the domain itself, that is the best solution.

“5  Another psychological issue is, at some point of time, the expert feels that he is revealing too much. He will no longer be a unique person with that level of knowledge once he shares that level of knowledge and the knowledge is placed in the public domain. The KE must also have that important point in mind and make sure that the expert does not feel like that as long as possible.”

Explanation system

The acceptance of the ES largely depends on its ability to explain its reasoning. The explanation is needed for the first reason that the expert domain is never perfect and it is quite possible that the ES decision is wrong.

When MyCin was developed its performance was found to be nearly 75% which the critics considered unacceptable. When human experts took the same test which MyCin was given, it was found be even lesser than 70. The point is, the expert domain is full of complexities and permutations, however hard you try, there is a possibility of a wrong decision. Thus when the decision is doubtful, the user may ask for explanation and decide whether to trust that decision or not. Humans also prefer to take second opinion when advised to be operated for example.

Another reason for providing explanation is that it is otherwise not possible for the human user to completely accept the decision as it would sound inhuman when the automated doctor says “you have a heart ailment” and cannot explain in human terms how it reached to that decision.

Earlier we have mentioned that there are multiple users of the system, from a normal user who has no knowledge about the jargon and an expert who expects a quick answer preferably in form of precise language of that domain. The explanation system also need to generate responses based on the user communicating with the system.

ES levels

There are many ways to categorize ES. One typical way is to see the level that the ES addresses. If ES is kind of helping an expert, for example when security expert ask for, it gives total number of packets with fragments, having source S, destination D, using a port number P as a source, etc, or give statistics about different types of packets base on some criteria like the application generated that packet, or the application which is target to those packets and all that. There are many such systems in many domains, and they enjoy a very good utility value; however they are most preliminary type of ES. We call this a helper ES.

A second level of ES is more independent and take decisions on its own. The experts interact with and judge its performance and advice for improvement. However independent such systems are, they are still under observation of other experts and they are ready to set things right when it appears that the ES is treading over a wrong path. We call this a peer ES.

In fact the most powerful, complicated and hard to build ES is one that can replace the expert himself. However, such ES is something which currently we only find in science fictions despite many serious research attempts. We call this a real ES.

Summary

Though generalized architecture is hard to provide, we have tried to see what common components an ES possesses. The ES has a query processor and client modelling part which takes the input and converts that input in to system understandable form. The ES has multiple types of users and this module models that user using the interface. There are many users which does not exist in conventional systems, for example two different types of experts; one for providing facts and rules and another for testing them in the final version. Additionally a learner is a user who learns from the ES. The ES decides the response and interacts with user based on his type. Knowledge storage and maintenance is another part which deals with the knowledge base and its schema. Most experts neither know how to deal with systems and provide input to it nor have time to learn so we need an intermediary known as Knowledge Engineer. KE has to learn the domain and also the system functions with the way the knowledge is organized in the system. Inference logic is applied over the knowledge to generate further knowledge. Knowledge store construction is just one phase. Expert knowledge is continuously updated. The ES must provide means for updating the knowledge using multiple possible ways. The ES cannot work without proper explanation facility if it has to deal with humans. There are three different types of ES, helping ES, peer ES and independent ES.