Information, communications and entertainment 21st century (ICE 21)-an Intelligent Home Centre

Citation

Martinez, A. (2003), "Information, communications and entertainment 21st century (ICE 21)-an Intelligent Home Centre", Kybernetes, Vol. 32 No. 3. https://doi.org/10.1108/k.2003.06732caf.002

Information, communications and entertainment 21st century (ICE 21)-an Intelligent Home Centre

Information, communications and entertainment 21st century (ICE 21)-an Intelligent Home Centre

Keywords: Cybernetics, Communications, Entertainment

AbstractOutlines innovative ideas for information, communication and entertainment in the 21st Century. Develops the concept of an Intelligent Home Centre, describing its actions and functions. Examines both the software and the hardware requirements and discusses both their role and development.

Introduction

The ideas contained in this article were first entered as a patent application in November 1997 with further additions up to the last application, which was entered in October 1999 and subsequently let to lapse a year later for lack of funds.

Looking back at the work that was done to update the original application, to the numerous inputs that were received from various sources and to the enthusiasm with which presentations were received, it may have been surprising at the time that we were not able to get any support from any company to fund the development of what was and still is a complex project.

We can only conclude that the concept is all too obvious, that the convergence of the developing technologies and the evolving market drivers will inevitably lead us to what is being described here.

This article is thus written as just another vision of what future generations could have to use and enjoy.

ICE 21

Technologies are emerging that can offer, together with existing systems comprehensive services in information, telecommunications and entertainment.

Currently, television receivers, video tape recorders, satellite set top boxes, hi-fi equipment, DVD players and the like are all sold as separate units, each in its own box or boxes. Each one of these devices is controlled separately leading to a plethora of interconnections, remote controls, power sockets, etc.

As new technologies continue to be developed, emerge and be adopted, our society will be facing greater and greater challenges to be able to use the services that will be available without having to spend an ever increasing time to understand, manage and be able to enjoy these services.

Today many people are still baffled by the complexity of some of the controls provided in many appliances e.g. setting a video recorder to record a future programme, or setting the intercarrier sound for a particular satellite transmission, or even re-setting the clocks after a power failure. How many parents purchase new equipment, may be on the advice of their more technically competent children, and are then left to unravel the complexities of the operating manuals. Have we not experienced the chaos of a visit from the grandchildren who playfully got back behind the television receiver and decided to unplug a couple of leads. You may even have five remote controls neatly placed on your coffee table in front of the settee so that you know exactly which one is which, but... where is number six... and kick-off is only 2 min away.

The equipment being used today contains a tremendous amount of redundancy in that the TV, set top boxes, hi-fi, etc. are sold as separate units each with their own power supplies, audio amplifiers, complex inter-connects, remote controls, power plugs, decoders and so forth. A great deal of this redundancy adds to cost and interconnect complexity.

It is therefore important that we consider ways in which we can overcome the constraints that we have identified thus far, by providing a cost effective, environmentally and user friendly, integrated and easy to use, intelligent, information, communications and entertainment centre, so that technology can be more easily accessible and enjoyed by everyone. If we were to overcome the existing constraints, future generations may even judge us to have been endowed with some degree of concern for having developed products that satisfy the real needs of users. After all one of the main purpose of engineering is to match identified needs to specific technologies that will yield cost effective and user friendly end products.

The essence behind this article is to propose a system, which is very easy to use, requiring no manuals to be studied and to be controlled by the user voice and a very simple remote control. Although, the concept was aimed at the home user in his living room, the software and hardware proposed in this system will have many other uses for the handicapped as well as in industry and commerce.

One often hears that the hi-fi enthusiast will never link his equipment to the television screen, but developments in DVD technology will lead to music being sold with images, whether it be pop, ballet, or orchestras and music only CDs could end up in the attic with the vinyls.

Equally, there is another school of thought which says that the PC will enter the living room and the TV theatre will be nothing more than a PC with a few more peripherals. With this vision in mind, one thinks of the manual worker who comes home exhausted, looking forward to a pint, his dinner and watching his favourite team on TV. Word processing and Lotus 1, 2, 3 may be somewhat away from his mind and if there was a PC in his living room it could have even more user redundancy built into it that the average PC has today.

Figure 1 shows a visual representation of a consumer today, keen to use the latest equipment available, trying to accommodate it all in his living room and to come to terms with the various manuals and remote controls.

Figure 1 Techology overload

Once he has managed to have everything up and running and to control the various appliances by keeping the manuals at hand, he may find that many of the facilities available do not get used very often and when required the manuals may have to be read again, if indeed they can be found.

How many times does grandpa gets a phone call from his favourite grand daughter with a request to record a particular programme from a given radio station. He looks at the clock and finds that there is only 15 min to zero time and he needs to have everything set up. The problem is that he has not used the tape recorder in his hi-fi for the last 7 months. He has to find the tape, set up the recording mode that was requested of him, tune the radio to the appropriate station and wait for the time to record, but how does he set up "record" and where is that manual.

Figure 2 shows the proposed "Intelligent Home Centre" offering all the required services to the consumer with one simple remote control.

The centre contains a screen, amplifiers, speakers, processors, various functional modules and connections to a single power point, aerials, telephone and cable.

Figure 2 Intelligent Home Centre

The proposed centre is easy to expand in order to accommodate the facilities required by the consumer. It does not require any technical knowledge to operate what we could call "great grandma proof". There are no technical manuals involved and only a very simple remote control.

The remote control as shown in Figure 3 contains a switch, which can be a capacitance switch so that it can only work when it is held in the hand. A microphone to receive the voice commands and only four buttons: UP, DOWN, YES and NO. The voice commands are modulated and transmitted to the centre via a RF or infrared link.

Figure 3 Remote control unit

The heart of the centre is a processor, which contains a novel command and control software and hardware architecture, the purpose of which is to receive voice and other commands from the user via the remote control, interpret these commands and carry out the necessary instructions. In order to avoid any possible errors, if the instructions could in any way be mis-interpreted, an audio visual feedback mechanism comes into play.

The screen will then display a series of questions supported by voice, (audio/ visual feedback) from the centre, the answer to each question can only be a yes or no. The corroboration from the user will then take place by pressing the YES or NO buttons in the remote control or by voice feedback. The centre audio/ visual feedback is also employed to guide the user in order that full advantage can be taken of any facility available if and when required.

In addition to the screen and the speakers, the centre contains a power supply, audio amplifiers, the command and control central processor and a series of functional plug-in modules. All the modules are interconnected to the power supply, the audio amplifiers and the central processor via high-speed busses. The module that connects the centre to the external display is specific to the type of display used.

One of the functional modules contains signal decoders to which module the outputs from other functional modules are sent in the form of audio/visual data streams. In this module, the format of each signal is identified, routed to an appropriate decoder and the corresponding audio and video outputs sent to the audio amplifiers and the video display module. Encrypted audio/visual data streams can also be dealt within this module by using the required keys. In this function, the central processor controls the routing of all the signals but does not perform any type of decoding, which in turn will avoid any interference between the processor clock pulses and the audio/visual signals, thus maintaining the highest level of signal quality.

To improve further the quality of the signals from the functional modules and to avoid any interference from the central processor, once a selection has been made by the user and the command and control processor have carried out the required selection, it will then switch itself into a dormant mode and will remain in that mode until a new command is sent by the user.

Equally the command and control processor will switch on the required functional module or modules to carry out a request made by the user, keeping all other modules not being needed in a stand by mode.

In addition to the power supply, audio amplifier and the command and control central processor, many different types of functional modules may be connected to the system including terrestrial TV, cable and satellite tuners, video and audio data stream decoders, DVD players, IDSL modems, hard disc recorders, PCMCIA readers, digital camera input module, "Blue Tooth" interconnect, etc.

The common features of the functional modules are that they can physically attach to the centre processor via the high speed busses, which will also connect them to the power supply and enable them to come under the command and control architecture.

Following a user selection, for instance to play a DVD, the central processor will carry out the required communication with other functional modules in order to direct the audio/visual data stream to the decoder module where an MPEG decoder will output a video signal to the video display module and an audio signal to the audio amplifier. The MPEG decoder in this functional module is common to all the functional modules thus avoiding duplication.

The audio amplifier is automatically set to the volume that was last used when a previous DVD was played. The user, however, can command the system via the remote control to "ADJUST VOLUME" and the system will display a colour bar at the bottom of the screen, which the user can adjust with the up/down button in the remote control. Alternatively the colour bar can also be adjusted with voice commands "UP,UP,UP or DOWN, DOWN". All functional modules not involved in this selection are switched to a stand by mode as well as the central processor, which goes into a dormant mode once the selection and adjustments are completed. Only the DVD player, the decoder module, the audio amplifier and the video module in addition to the display are operational when a DVD is playing.

When a new module is plugged into the system it will be recognised by the software architecture enabling the user to operate it from the remote control.

If the user is not familiar with the operation and facilities offered by the new module it can command the central processor to "INSTRUCT" and the system will go into the audio/visual feedback mode informing the user of the facilities available and how to enjoy them.

Voice signals received by the remote control are transmitted to the command and control module where they are translated into signals, which are sent to the appropriate functional modules via the high-speed busses. A command such as "SATELLITE TELEVISION SPORT CHANNEL ONE" will be easily understood by the system and the appropriate selection made of both the tuner module and the tuning frequency needed to display the requested channel.

In addition to the above, the audio amplifier will be adjusted to the volume which was last used when this particular channel was selected. This will also apply to the brightness and contrast in the video module.

If the command and control module is unable to interpret any particular voice command, it will move to the audio/visual feedback mode requesting specific clarifications to which the answer in each case is either "YES or NO" only. An example of such clarification may be: "Do you wish to watch television", "Do you want to listen to some music", or "At this time you normally listen to the news program on channel one, is this what you want to watch".

In order to be able to offer the above facilities the command and control architecture will have a high degree of intelligence and intelligent agents will need to form part of the overall system if it is to achieve the required ease of use.

Bradford University, which have been carrying out a great deal of research in intelligent agents proposed the software architecture shown in Figure 4. The main purpose of the intelligent agents is to enable the system to perform complex functions from very simple commands.

The software architecture proposed allows a collection of hardware functional modules to be controlled using a user-friendly high level interface. To achieve this the intelligent agents work both independently and in collaboration in order to perform user directed tasks. In addition to the agents, the software architecture employs knowledge-based techniques in order to control and direct the agents.

Figure 4 Software architecture block diagram

The architecture is layered from A to G. Layer A interfaces with the user and contains the highest level of intelligence. Layer G is a hardware layer, which controls and is controlled by the functional modules. The user interacts with layer A by requesting information and being presented with diagnostics, help and status data. New functional modules are added at level G by plugging them into the system. Each functional module contains enough information describing how it can be controlled by the central command and control architecture.

The software is therefore, layered and intelligent processing occurs at the upper layer leaving the lower layers to control the functional modules.

Knowledge based system technology is used in the upper layers to implement an inference engine and a knowledge base that will control the behaviour of the agents and ensure that the user is presented with the necessary information in the most appropriate fashion.

Modular architecture is proposed with object-oriented software technology so that when a new functional module is plugged into the system, a new agent is created within the lower layers.

With this software architecture the system is expandable. Each functional module requires an interpreter and driver, which translates the intelligent processing from the upper layers into the control data required to control such a module via the high speed busses. When a new module is added, it is only necessary to add the corresponding driver and interpreter without the need for modifications or additions to the upper layers.

In layer A, the user input is evaluated by the inference engine and translated into queries, which involve the intelligent agents. These queries are then passed to the agent module controller, which co-ordinates the use of agents to satisfy the queries.

In layer B, a collection of agents are used to control the functional modules with each module having a corresponding agent. This means that only agents, which correspond to a particular module, need to be present. When a new module is added to the system only then is the corresponding agent added to this layer. The "Intelligent Agent Module Control" in layer A passes queries to the agents. Each agent is a separate software component, which will attempt to satisfy a query using its own local information and by interacting with other agents.

In layer C, the object level system specifies the behaviour of each of the hardware devices at a high level of abstraction. It is therefore responsible for implementing the hardware devices within the common high level object- oriented language.

In layer D, the architecture uses virtual machine technology. This allows the system to be incrementally expandable to meet the user's needs. The language used to implement the intelligent agents will be developed using a virtual machine. This layer defines the operation of the language and also the external interface, which is required by the hardware controllers.

In layer E, each hardware functional module has an interpreter required to translate information at the virtual machine interface, which in turn, is then used to drive each module.

In layer F, each functional module has a driver representing the physical interface between software and hardware.

In layer G, each hardware functional module is introduced to the system when it is plugged-in to the high speed busses. Each module containing a unique identifier, that is processed by the system enables the appropriate software to be added and the module in turn to be controlled when required by the user.

Processing therefore occurs as a result of a user command, this is then translated into a sequence of low-level commands and sent to the drivers. This translation takes place as a sequence of transformations as follows.

1. 1.

   The user input is first tokenised to produce a stream of atomic tokens, which the system can interpret.

2. 2.

   The tokens are then parsed with respect to a command grammar to generate structured commands. The command grammar being the reference for associating the structure commands with a series of tokens and the parsing being the process carried out in software, which, given a grammar and a sequence of tokens, recognises and produces a series of commands.

When a functional module is plugged into the high-speed busses, the software associated with it may invoke a change to the system command grammar and provide additional structured commands enabling access to the features offered by the new module.

If conflicts arise as a result of parsing ambiguous commands the system can then use its audio/visual feedback to clarify the user's command. Some conflicts may be resolved by taking into account user preferences with information available in the system knowledge base.

With the proposed architecture, a high level structured command can be understood by the system, whereas the voice input itself is not. Each high-level command can be complex and may require many low-level device commands to be executed. The first step in carrying out a high-level command is to transform it into a collection of messages, which are sent to the intelligent software agents controlling the functional modules. This transformation would need to be performed by the Knowledge Base System, encoding knowledge about how to achieve high level commands. Knowledge is thus context sensitive since it would depend upon the available modules, information from sources such as the Internet and user profiles. The available knowledge in the system is encoded as a collection of rules and facts, facts being always true and rules define how to deduce new collection of facts when it is known that another collection is true.

The system knowledge associates agent commands with high level commands and as new modules are added then the knowledge within the system is also updated.

Agent commands are sent to an agent controller, which acts as a message broker for a collection of independent intelligent agents. An intelligent agent is an independent entity consisting of state and behaviour, the stage being a collection of data. For example, the agent controlling the display screen will contain a state component in order to describe whether or not the television tuner is currently switched on. The behaviour of an agent is a collection of handlers (programs) being activated when the agent sends a message from the message broker, with each handler having access to an agent's state. An agent will need to have a message handler for each command sent to it. In addition an agent will also need to support other message handlers since agents can communicate with each other via the message broker. As an example, the satellite receiver agent will need to know whether or not the display screen is switched on.

Some agents will need to be designed to introduce new information into the system. For example, the agent responsible for monitoring broadcast programme schedules published on the Internet will need to update the Knowledge Base as information becomes available.

The additional software required when a module is added could be obtained by the system without interaction by the user. This could be achieved by downloading the required software from a dedicated Web page supported by the manufacturer of the equipment. Once downloaded, the additional software would be integrated into the system, which will then inform the user via the audio/visual feedback that the new functionality is available providing also the required instructions for its full use.

The architecture outlined above is just an example of how an intelligent system such as the one proposed could achieve a very high level of user friendliness and it would not exclude other alternative solutions as the confluence of new software and hardware developments will support future requirements from end users.

The proposed concept does not deal with the specification of the busses, which would need to be developed in line with the proposed modules in order to satisfy a particular market.

New development in FLASH technology, DVDs, "Blue Tooth", and hard disk will impact the design of modules to satisfy consumer product within the above concept proposals and developments in flat panel displays will lead to systems being supplied as a separate unit in addition to the speakers.

Simple systems can be developed today with a relative low level of intelligence that may only require a few hundred voice commands to operate and will still avoid a great deal of hardware duplication and redundancy. More complex systems with higher intelligence and knowledge base capabilities are natural developments of the above concept.

Functional modules with edge connectors plugged into an equipment rack are one of the many possible hardware solutions proposed. Alternatively functional modules can be designed, which can plug into other modules and being capable of continuously expanding the system without the use of an equipment rack.

I would like to acknowledge the inputs, the support and encouragement that I received from the computer department of the University of Bradford and from my colleagues in Chase Advanced Technologies without whom this article could not have been written.