Movies
video1
video2
video3
video4
video5

Related Websites
Virtual Reality Lab at Buffalo

Details regarding implementation

Abstract

An interactive virtual factory test-bed dubbed the VR-Fact was developed as a part of a research to explore the applications of Virtual Environments (VE) in the area of manufacturing automation. This fully immersive factory building tool can be used to createl Virtual Factories using modular machines that a designer can drag and place in the factory to study issues such as plant layout, clusters and part flow analysis. This paper covers several issues of developing such factories including the factory layout strategies and scene graph creation techniques. Mathematical clustering algorithms were implemented to apply traditional factory design methods to automatically generate complex virtual environments. Such algorithms can also be used in other feature based applications to self-generated virtual environments.

1. Introduction

In this paper, VE applications in the area of manufacturing are explored. One natural area of application is in simulation of machining, casting and simulation of plants in the process industry. It is a commonly accepted view that this area has not fully benefited from the true capabilities of Virtual Reality systems. Immersion, interaction and real time computational issues are exciting research topics which need to be pursued more vigorously to achieve this goal. To investigate all of these issues and to develop this technology further, a test bed called a VR-Fact was developed at the Virtual Reality Laboratory of University of Buffalo. The significance of this work is that this medium allows testing of product design and the factories that build them without having to manufacture a real part until it is ready for the market or for a particular customer.

2. Background

The most common use of VE in engineering is in design visualization and prototype evaluation. Ford (Mahoney, 1995), Chrysler (Hamilton et. al. 1992), John Deere (Owen, 1994), NASA (Hancock, 1993) and Northrop (Hamilton et. al 1992) are developing virtual prototyping environments for ergonomic assessment of preliminary designs. These development work have shown encouraging results for the use of VE.

In the area of manufacturing, VE is developing into an important area of research. Traditionally topics such as simulation of manufacturing processes and process plans (Lu, 1996), special interfaces for CAD/CAM systems, analysis of manufacturing layout (Banerjee, 1996), training and education are some of the areas of research. The focus of the present work is to integrate a full product cycle in a Virtual Factory, formalizing the various standards of virtual manufacturing with-in this factory and developing a modular approach to developing a complex system from smaller autonomous units.

3. Concept of a Virtual Factory

UB VR-Fact environment is based on three principles:

• Model before you build
• Simulate before you produce
• Anticipate and solve before they occur

The virtual factory described in this paper is focused in three levels of details. The top level is the factory implementation with machines, transportation, control and scheduling models. The intermediate level is the machine level implementation where each machine can be controlled and loaded with parts and appropriate codes. In such a model, it will be possible to "touch" the machines, control "knobs" and see actual functionality. At the micro-level, the VR-Fact will integrate process simulations such as metal chip formation with given definition of tool geometry, simulation of chatter, tool wear etc. This step will also foster design for manufacturability analysis and design evaluation for a given assembly or manufacturing process.

The present paper describes the factory level implementation only. In this level, the factory consists of several modular machining centers, milling, turning, grinding, drilling and coordinate measuring machines. The factory also has robots, conveyors, automated guided vehicles and human operation centers such as assembly, finishing and inspections stations. The following research topics have been addressed in this research to make this factory as seamless and practical as possible:

Modeling of the factory: The factory modeling was done on Pro-Engineer and the virtual interface was developed in the World Tool Kit virtual reality software.

Factory design: Computational and algorithmic modules for design and placement of machines in the factory was developed.

Machine grouping algorithm and scenegraph creation: Algorithms for grouping machines and interface for inputting process plan was developed. The outputs are saved as scenegraph to generate the Virtual Environment.

Natural interface with the factory: The interface with factory is provided through several Virtual Reality hardware devices including stereo Head Mounted Displays (HMD), instrumented gloves and software interface to grab, drag and navigate in the factory.

The following sections discuss in detail some of the topics described above.

4. Modeling the Factory

The modeling of the factory which constituted a significant part of the development of the virtual factory was carried out on the Pro Engineer CAD package. Within Pro/E the models were saved in slp format using a few basic criterion dictated by its future use within the World Tool Kit modeling software. The location of objects within the factory and the coordinates needed special attention before exporting the geometry.

5. Cell Formation and Scene Graph Creation

Design of factory it self can be carried out using many different philosophies. Here we have experimented with one system of factory design called the Cellular Manufacturing System which deals with the problems of classifying parts and machines into part families and machine cells. In shopfloor design of this type one can take advantage of the process similarities in manufacturing for achieving economical benefits similar to mass production. There are three different methods of identifying machine groups: Visual inspection-based, part characteristic-based and production process-based methods. The most efficient way of finding machine groups is the production based-method. Groups of parts are divided into families by analyzing the production flow (PFA) according to the information in the routing sheets or process plans. PFA was originally developed as a manual procedure and is based on the premise that the majority of components and machines belong to clearly defined family of cells. PFA leads to group technology (GT) organization. It does not change the plant design, product design, process methods or sub-optimization process such as cost minimization at this stage. We have experimented with several classical algorithms of shop floor design. These are some of the methods that we have implemented (Lefort and Kesavadas, 1998, Singh, 1996):

1. Single Linkage Cluster Analysis using Similarity Coefficients between machines
2. Modified Boolean Matrix Method (MBM)
3. Bond Energy Algorithm (BEA)

More details regarding the above methodologies

Visualization of complex part-machine matrix using Cellular Manufacturing System

Once the machines and the factory structures are modeled, one of the most challenging issue is the process of creating the Visual representation of the factory based on the results obtained from the cell formation algorithms. To create interaction the user has to have the abiltiy to navigate within the factory as well as pick single or group of machines in real-time. We developed an automatic scene graphing algorithms to achieve this process.

The machines and all of the interaction parameters were modeled as nodes and hierarchies were defined to configure tha factory automatically. The development of the structure of these nodes and their hierarchy was one of the most critical parts of this research. All subsequent design of the factory, checking of intersections and the flexibility with which the factory can be reconfigured are essentially a function of the structure of the node hierarchy.

General organization of the nodes

The root node of the factory was created in the main program. The nodes root and limit are the only two nodes that are declared as global variables. That is because the root has to be created in the main program and limit is used in other routines like the one which test intersections. The rest of the nodes such as the geometry nodes, transformation nodes, separator nodes and group nodes are defined within the routine createscenegraph().The root is the parent of all other nodes. Now we define two main families of nodes. The first composed of geometry, non-movable entities and the ones that do not have to be subjected to intersection testing (figure 1 and 2) and the second composed of the movable geometry (machines, robots etc) (figure 3). This is done to partition the world and allow intersections and selection during the simulation. As an example (figure 3) a group node, called limit is created which is the parent of all movable nodes.

.

All non-movable parts and light nodes, which do not have to be tested, are attached to the root node. When testing for intersections we start from the limit node and traverse down the sub-tree hierarchy testing each node containing geometry.

Now once the Cell Formation algorithm finds a group we immediately store the information within a limit node called cell 1, cell 2 etc (figure 4). In the group let's say the algorithm returns four machines (milling, drilling etc), they are placed at four different positions. We define the first geometric transformations using vectors, quaternion and 4x4 homogeneous matrices. Below the separator node for cell 1 the WTK Transformation State will be the transformation values assigned to the transform node cell 1. Only the geometry of the cell 1 defined right after that will be affected by this state and not the sub tree above it. Thus the utility of the cell 1 separator node is that it prevents this transformation state to be transmitted to the next machine cell 2. The idea is that the separator node creates a protective bubble inside of which the transform node can play its role without interfering with the rest of the world. These concepts of state of propagation and state accumulation are valid for transformation, lighting and fog states also. We also use the concept of Object Oriented programing to catagorize the cells. Thus each cell derives independant properties for all further processing of cells. This approach reduces the time of developing complex scenarios. In the future we plan to use the object oriented paradigm to include other factory activities also.

6. Virtual Factory Graphical User Interface

The biggest advantage of using VR-Fact is the interface for inputting machine, part information through a 3-D graphical user interface. Since aspects such as space, logistical and technical limitations are not considered in the algorithms, the user is provided with the ability to interactively relocate the machines after the computer has placed them. In this research, World Tool Kit^® (WTK) Graphic Utility is used to create the GUI to simplify the interaction with the user.

The virtual reality program consists of three windows: a main immersive view window, a top view window and a message window (Figure 6). Furthermore, several pop-up windows and pull-down menus for important commands, messages, and interactions are used. To make the program user-friendlier, an interactive GUI for the database is added, where the user can add, remove and check machines, parts and operation data directly by using the virtual matrix.

Figure 6: Three windows are implemented in the GUI to make the program user-friendlier: The "normal" view window (Main window) gives the user the impression and feel of walking around in a normal factory. The top view window is preferred to view the machine cells and to relocate machines for customize the shop floor layout. The message window is an effective way to interact with the user.

While customizing the factory layout, the user can select any machine from both the viewpoints and select machine group to relocate them wherever desired. A white wire-frame box indicates selected items that the user can incrementally move around by using the arrow keys or by drag and drop operation using the mouse. The interaction capabilities described above leads to a powerful new way of designing and customizing a factory layout. An example of designing the factory using this tool is show in the movie clip attached here.

More images of the virtual factory

7. Conclusion and Future Research

A virtual factory test-bed called the VR-Fact was designed and implemented in the Virtual Reality Laboratory at UB. The software was implemented on a Silicon Graphics ONYX 2 computer and can be immersively viewed using a head-mounted display or a Crustal Eye stereo glasses. This factory can be used to interactively design shopfloor arrangements for a given product mix and set of machines. While building an actual factory may take as long as 6 months to one year, we can test it using the VR-Fact environment in a matter of few weeks. Mathematical algorithms implemented here give the designers cues to good grouping of machines for the given task which helps in testing out various scenarios in advance. Future research in this area will be focused in encompassing larger issues of virtual manufacturing including better algorithms for optimizing factory layout, real time simulation of factory operations and the use of internet based distributive control of these virtual systems.

References

Banerjee, A., and Mehrotra, S.,1996,"An Enabling Environment for Inputting Qualitative Information in Manufacturing Systems Layout Design". Proceedings of Symposium on Virtual Reality in Manufacturing Research and Education, University of Illinois at Chicago, pp 62-69

Carolina Cruz-Neira, C., 1993,"Virtual Reality Overview", ACM SIGGRAPH '93 Course Notes: Applied Virtual Reality, ACM SIGGRAPH '93 Conference, Anaheim, California, August 1-6.

Durlach, N.I., and Mavor, A.S., 1995, "Virtual Reality: Scientific and Technological Challenges", National Academy Press, Washington, D.C.

Hamilton, J., E.T. Smith, G. McWilliams, E.I. Schwartz, and J. Carey, 1992, "Virtual Reality, How a Computer-generated World Could Change the Real World", Business Week, October 5,pp. 97-104.

Hancock, D., 1993,"Prototyping the Hubble Fix", IEEE Spectrum, October,pp. 34-39.

Hubbard, P.M., Program of Computer graphics, Cornell University, "Real-time Collision Detection and Time-Critical Computing", Proceedings of SIVE95.

Lefort.L and Kesavadas, T "Interactive Design of Virtual Factory using Single Cluster Analysis", Presented at the IEEE International Conference on Robotics and Automation, Lueven, Belgium, May 14-19, 1998.

Lu, W.,1996, "Graphical Simulation for Process Planning in Virutal Manufacturing". Proceedings of Symposium on Virtual Reality in Manufacturing Research and Education, University of Illinois at Chicago, pp 153-162.

Mahoney, D.P., 1995,"Driving VR", Computer Graphics World, Vol. 18, No. 5, May, pp. 22-31.

McCarthy, W.D., Sheasby, S., Amburn, P., Stytz, M.R., and Switzer, C., 1994, "A Virtual Cockpit for A Distributed Interactive Simulation", IEEE Computer Graphics and Applications , January, 49-54.

Narayanan, S., Dech, F., Banerjee, P., DeFanti, T., 1996, "Protocols for Integrating VRML in a Virtual Manufacturing Environment", Proceedings of Symposium on Virtual Reality in Manufacturing Research and Education, University of Illinois at Chicago, pp138-142.

Nelson, B., 1996, "Assimilating Disparate Sensory Feedback Within Virtual Environments for

Owen, J.V., 1994, "Making Virtual Manufacturing Real", Manufacturing Engineering, Vol. 113, No. 5, November, pp. 33-37.

Singh, N.,1996 " System approach to Computer Integrated Design and Manufacturing ", Wiley&sons.