Potential identification and industrial evaluation of an integrated design automation workflow

Abstract

Purpose

The paper aims to raise awareness in the industry of design automation tools, especially in early design phases, by demonstrating along a case study the seamless integration of a prototypically implemented optimization, supporting design space exploration in the early design phase and an in operational use product configurator, supporting the drafting and detailing of the solution predominantly in the later design phase.

Design/methodology/approach

Based on the comparison of modeled as-is and to-be processes of ascent assembly designs with and without design automation tools, an automation roadmap is developed. Using qualitative and quantitative assessments, the potentials and benefits, as well as acceptance and usage aspects, are evaluated.

Findings

Engineers tend to consider design automation for routine tasks. Yet, prototypical implementations support the communication and identification of the potential for the early stages of the design process to explore solution spaces. In this context, choosing from and interactively working with automatically generated alternative solutions emerged as a particular focus. Translators, enabling automatic downstream propagation of changes and thus ensuring consistency as to change management were also evaluated to be of major value.

Research limitations/implications

A systematic validation of design automation in design practice is presented. For generalization, more case studies are needed. Further, the derivation of appropriate metrics needs to be investigated to normalize validation of design automation in future research.

Practical implications

Integration of design automation in early design phases has great potential for reducing costs in the market launch. Prototypical implementations are an important ingredient for potential evaluation of actual usage and acceptance before implementing a live system.

Originality/value

There is a lack of systematic validation of design automation tools supporting early design phases. In this context, this work contributes a systematically validated industrial case study. Early design-phases-support technology transfer is important because of high leverage potential.

Citation

Entner, D., Prante, T., Vosgien, T., Zăvoianu, A.-C., Saminger-Platz, S., Schwarz, M. and Fink, K. (2019), "Potential identification and industrial evaluation of an integrated design automation workflow", Journal of Engineering, Design and Technology, Vol. 17 No. 6, pp. 1085-1109. https://doi.org/10.1108/JEDT-06-2018-0096

1. Introduction

According to Ehrlenspiel et al. (2007), around 70 per cent of the costs for the market launch of a manufactured product are defined in the very early phases of the product life cycle. Thus, the design and manufacturing industry strives to pull decision points of high predictive quality upstream in the overall product creation process, increasing the pressure on the development departments to deliver faster, better and cheaper products, and thus to provide a competitive advantage for the organization (Baxter et al., 2008). Design automation (DA) has already been identified as a key enabler for addressing these challenges (Rigger et al., 2016). Two main leverages are used: On the one hand, automating routine and repetitive design tasks saves time and costs, and can be seen as achieving increased time for earlier stages of design processes (Skarka, 2007; Verhagen et al., 2012). On the other hand, supporting the early design phases by automatically exploring large and unstructured design spaces (Dym and Brown, 2012) and generating alternative design solutions (Chakrabarti et al., 2011) leads to improved product quality and shortened lead times (RQ2 in Section 5.1). However, there is a discrepancy between the availability of DA methods and their industrial application, especially regarding computational support of early design phases (Rigger and Vosgien, 2018). Reasons are uncertainties with respect to the awareness of available opportunities, recognition of potential of applying DA and ability to define the automation task (Bolognini et al., 2012; Rigger et al., 2016) (RQ1 in Section 5.1).

Thus, the motivation of the presented work is to help to increase awareness, understanding and adoption of DA solutions for the early stages of design processes, as such a support is still extremely under-leveraged in industry.

The paper contributes by presenting a systematic validation of design automation applications in design practice with a focus on evaluation of design automation opportunities for the early stages of the design process. To communicate potential for novel design automation applications (here early-stage design task automation), we put into context novel technologies with already existing ones (here later-stage design task automation).

We introduce an industrial case study concerning the design of ascent assemblies at Liebherr-Werk Nenzing GmbH (LWN). Two example cranes with ascent assemblies are shown in Figure 1. The DA tools used in the case study are a prototypical design space exploration tool generating a number of pre-optimized layout designs (Zăvoianu et al., 2018), and a product configurator, which is in operational use at LWN, to perform assembly configuration tasks mostly occurring during the detailed design phase (Frank et al., 2014). We approach the case study by developing an automation roadmap, i.e. modeling and comparing as-is and to-be processes:

• without any DA tools;

• with the product configurator only; and

• with the integrated product configurator and design space exploration tool (referred to as integrated workflow).

The success of the in-operational use product configurator (i.e. posterior to implementation) is validated using quantitative and qualitative assessments. For the prototypically implemented design space exploration tool, as well as the integrated workflow, a potential estimation and qualitative assessments including acceptance and usage prior to implementation are performed.

This paper argues that a combination of early and later process stage DA tools can, on the one hand, effectively meet typical industry requirements for DA, such as the reduction of lead times and error rates. On the other hand, the integration of a tool which generates a number of alternative layout designs (here crane ascent assembly paths) with a tool which automatically creates the detailed CAD ascent assembly models and respective production costs, bears the potential to evaluate and compare alternative, potentially novel optimal design solutions, and hence, to pull the decision-making to a relatively early point in the design process (again RQ2 in Section 5.1).

Section 2 positions our work regarding the current state of the art of DA and the limited application of DA solutions. Section 3 describes the approach for identification and estimation of DA potential before implementation and for industrial success evaluation after implementation. In Section 4, the case study for integrating DA solutions supporting the design of crane ascent assemblies in the early and later design phases is introduced. The results and the validation of the industrial evaluation of the two DA solutions as well as their seamless integration are presented in Section 5. In Section 6, these results as well as the potential to extend the introduced approach to a structured methodology for potential identification and success validation of DA solution are discussed. Finally, the paper is concluded in Section 7 with a summary and an outlook on future work.

2. Related work

2.1 Design automation

Two major communities of DA research can be identified, namely, knowledge-based engineering (KBE) (La Rocca, 2012; Verhagen et al., 2012; Stjepandić et al., 2015) and computational design synthesis (CDS) (Antonsson and Cagan, 2001; Cagan et al., 2005; Chakrabarti et al., 2011; Chakrabarti, 2013). Both investigate computational approaches to support design tasks by means of automation, and aim at improved reliability of predictions about states and features of future products and processes, before making decisions with a high impact on committed costs or other objective functions.

KBE approaches focus on the automation and streamlining of routine/repetitive design tasks predominantly occurring in later design stages and have their origins in knowledge based systems (Dym and Brown, 2012). Such design tasks are commonly characterized by pre-structured solution spaces as, e.g. often occurring in the context of adaptive design (Pahl et al., 2007). KBE methods and tools tend to capture and deterministically automate already known best practice designs and can thus be termed conservative in nature.

CDS approaches aim at supporting early stages of design processes and can be termed systematically explorative in nature. Typically, large and often unstructured solutions spaces, such as more commonly occurring in the context of original design (Pahl et al., 2007), are explored using stochastic search strategies to generate a number of alternative, potentially novel designs. Thus, CDS aims at enabling and encouraging engineers to realize optimized designs beyond bias and by thinking out of the box. Furthermore, solution alternatives can be analyzed, compared and traded-off according to various design criteria, which can improve solution quality. As CDS methods are applied to support early design process stages, improved solution quality has potentially a very high impact.

2.2 Limited industrial application of design automation

The application of DA in industrial product development processes is still mostly limited to routine design (Verhagen et al., 2012) and redesign (Tomiyama, 2007) tasks, including the relative widespread use of configurators (Zhang, 2014; Willner et al., 2016). On the contrary, with a few exceptions only, successful applications aiming at design support in the early design stages remain limited to academic demonstrators validated by means of fictive case studies; industrial applications and evaluations are virtually absent (Bolognini et al., 2012) (again RQ1 in Section 5.1). Verhagen et al. (2012), Tomiyama (2007), and Bolognini et al. (2012) overview main reasons for DA project failures. By the presented case study we contribute to overcoming the following two reported reasons.

Firstly, systematically assessing a DA solution before and after its implementation is still a critical challenge for both industrialists and scientists. This is a key issue to lay the foundation for comparability, benchmarking and determination of return on investment (ROI) (Verhagen et al., 2015). Some guidelines on potential identification and justification are available (Stokes and Consortium, 2001; Emberey et al., 2007; van der Velden et al., 2012), but they are solely qualitative in nature, tend to focus on technological aspects, and are rarely and inconsistently applied (Verhagen et al., 2012). A first detailed approach to objectively quantifying automation opportunities for life cycle engineering tasks was introduced by Verhagen et al. (2015). The capability to assess and validate the added value of the deployed solution after implementation has been demonstrated by several case-based assessments (Shea et al., 2005; Singh and Gu, 2012; Emberey et al., 2007). However, there is still a lack of research on the practical implementation and adaptation in industry (Nordin, 2017). For such assessments, qualitative, quantitative, or mixed research methods can be used (Creswell, 2009). While engineers are more familiar with quantitative methods, qualitative ones can yield new insights, going beyond the findings achieved with quantitative methods (Daly et al., 2013), e.g. eliciting potential benefits as perceived by end-users (Hamraz and Clarkson, 2015). The combination of both prior and posterior evaluation, as well as qualitative and quantitative assessments is still open to be demonstrated.

Secondly, a particularly relevant reason of DA project failure mitigated in the context of this work is that DA often tries to achieve too many things at the same time, such as parametric design, optimization, data integrity management, process planning, and synthesis. As to this issue, it has been repeatedly noticed that besides a careful evaluation to find the right balance between manual and automatic design tasks, stepwise automation of well-defined design-process parts is often superior to overall full automation at once (Dym and Brown, 2012). This usually contributes to improved maintainability, re-use and adaptation of incorporated knowledge, and sustained usefulness and usability, because users can use such DA tools in more flexible ways (RQ5 in Section 5.1). Additionally, the application and combination of different types of DA methods can be desirable. Recent approaches have mostly focused on combining a synthesis or configuration task with an analysis task to generate validated design alternatives of product architectures (Münzer and Shea, 2015), optimize topologies (Cui and Wang, 2013), or automate the generation of simulation models for calculated design configurations (Johansson and Elgh, 2013; Colombo et al., 2015). However, the combination of DA methods of early and later design phases to combine potentials and strengths of the methods has not yet been demonstrated and validated in industrial settings.

2.3 Identification of design automation potential and industrial validation

As mentioned in the previous subsection as the first barrier for implementing DA methods, one crucial aspect of DA projects is the identification and justification of the planned development effort prior to the actual implementation, adaptation or combination. This works towards being able to calculate ROIs of DA projects (Verhagen et al., 2015). However, demonstrating the added value of a DA solution before and after its implementation is still a critical challenge for both industrialists and scientists. Some guidelines on identification and justification are available (Stokes and Consortium, 2001; Emberey et al., 2007; van der Velden et al., 2012), but they are solely qualitative in nature, tend to focus on technological aspects, and are rarely and inconsistently applied (Verhagen et al., 2012), so not widely perceived to be useful.

Verhagen et al. (2015) introduce a method for the identification and justification of DA opportunities through quantification of information waste. The method is the first detailed approach to objectively quantify automation opportunities for life cycle engineering tasks.

Another important aspect of DA projects is the capability to assess and validate after implementation the added value of the deployed solution. Even though many studies have aimed at validating the usefulness of generative design systems through case-based assessments (Chau et al., 2004; Shea et al., 2005; Singh and Gu, 2012), there is still a lack of research on the practical implementation and adaptation in industry (Nordin, 2017).

One noteworthy approach is presented in Hamraz and Clarkson (2015), where it was preferred to conduct structured interviews for the qualitative assessment of their solution and to underline potential benefits as perceived by the end-users.

Unfortunately and as already mentioned by Verhagen et al. (2012), “these approaches toward the assessment of the suitability for automation of engineering tasks suffer from a number of shortcomings”. For instance, assessment criteria are often arbitrarily defined and it is not clear how these criteria are to be used in practice. Furthermore, the proposed criteria only allow for a qualified assessment of suitability for automation and usability, and adoption criteria are often omitted because they are difficult to define and assess. The present research work claims to demonstrate the opposite. Moreover, none of the mentioned related works address both the potential identification and the solution assessment posterior to implementation. Finally, in most of the cases the assessments are performed in a certain context for a specific case study and do not consider the assessment of the solution suitability in other contexts (e.g. preliminary vs detailed design, original vs adaptive design).

3. Approach

Our approach, illustrated in Figure 2, is divided into two main blocks: Block A is dedicated to the potential identification and assessment of DA tasks prior to implementation. Block B addresses the industrial assessment and validation of DA solutions posterior to implementation. The overall approach is based on standard analysis and design processes of workflow modeling towards process re-engineering (Sharp and McDermott, 2009). The four steps are explained below:

• (I)  In the first step, the as-is process of the design task at hand is established. The process can be derived from existing documentations and/or established from scratch by performing specific workshops with the engineers.

• (II)  In the second step, the corresponding to-be process is modeled.

• (III)  By comparing as-is and to-be processes and by using quantitative and/or qualitative assessments, the potential and expected benefits of DA solution are elaborated. For the quantitative assessment, metrics, such as lead-time reduction, can be derived from a list of DA drivers defined in Rigger and Vosgien (2018). Similarly, qualitative evaluation with end-users, e.g. by conducting structured interviews can be performed according to this list of motivational drivers, overall contributing to consistency of criteria. Generally, a qualitative assessment often complements a quantitative evaluation. In particular, qualitative evaluation is often used in exploratory parts of a study, whereas quantitative measures are often more appropriate for directed and more specific evaluation, e.g. in the context of results about acceptability and usability. Finally, an automation roadmap is established, summarizing the DA drivers and the gaps between the as-is and to-be processes.

• (IV)  Once a DA application has been deployed as a productive system (i.e. in operational use), the success validation can be performed.

Two ways of validation are distinguished:

• (IV.1)  Quantitative assessment: Here, the same metrics as already applied for potential estimation (Step III) are re-evaluated and compared.

• (IV.2)  Qualitative assessment: By conducting structured interviews and questionnaires with end-users, the process improvement is assessed and feedback, e.g. regarding usability and acceptance is gathered.

4. Case study: integrating design automation solutions for the design of crane ascent assemblies

Figure 1 shows an offshore crane and a gantry of a mobile harbor crane manufactured by LWN. The ascent assemblies, colored in red, are the external access structures required to reach certain points/areas, e.g. for maintenance and steering. These points are termed access points. The composition of such ascent assemblies typically involves a set of standardized, parametrizable components (e.g. platforms, ladders, stair cases).

On the one hand, offshore cranes are standardized products realized in different variants. During the design phase, the re-use of components across these variants is ensured and the configuration and positioning of the ascent assemblies is subject to very few degrees of freedom. Updates to the design of crane and ascent assemblies are realized in regular, relatively long intervals. On the other hand, gantries are made of configurable main components, which are adapted according to the requirements of each customer, and the configuration and positioning of the ascent assemblies is subject to many degrees of freedom. Therefore, the potential for computational support of configuration and optimization will be studied in the context of original (offshore) and adaptive (gantries) design, respectively. RQ4 will be evaluated in this context.

4.1 As-is and to-be processes

Following the approach introduced in Section 3, first, as a baseline, the current as-is process was modeled separately by two persons from the technology management department. No major differences occurred. The modeling of the current as-is process was conducted again with all six engineers participating in the interview study. Again, no noteworthy differences were found, as this a well-defined, stable standard process. It includes the product configurator “Automatic Crane Component Design” (ACC-Design), which is in operational use and automates the rather repetitive and time consuming task of designing individual ascent assembly modules. Details can be found in Frank et al. (2014), a short summary is given in Section 4.3.1. Figure 3 (a) shows the current as-is process representing the description of the process steps, their sequencing, the input and output as well as the involved stakeholders.

The main inputs of the process are crane geometry, customer requirements and constraints, and existing standardized solutions of ascent assemblies. The final outputs are the “as-built” crane model including all geometrical and manufacturing details and the complete ascent assemblies and interface components between the ascent assemblies and the crane structure.

The process is decomposed into design phases known as preliminary (early) and detailed (later), where the preliminary one ends with the delivery of a final draft of a parametrized CAD model of the crane including preliminary ascent assemblies.

In the preliminary design phase, the processes for original and adaptive design differ: While for the original design, the engineers iteratively refine the crane design and adapt the ascent assembly components using ACC-Design, in the adaptive design, they sketch several and select one feasible layout for the ascent assembly and configure it based on this sketch in ACC-Design. In both cases, to verify and validate the generated ascent assembly solution, the engineers perform a clearance analysis in the CAD system to correct potential errors and inconsistencies in the model again using ACC-Design.

The detailed design process is the same for both adaptive and original design. First, the assembly CAD model is refined and completed with the appropriate standardized interface components (welding plates, bolted flanges, etc.) for mounting the ascent assemblies to the structure of the crane. Second, the manufacturing drawings are automatically generated in ACC-Design. The final step consists of choosing, refining and integrating the gratings of the various platforms.

The current as-is model is similar to the initial as-is model with the exception that the developed product configurator (ACC-Design) has been integrated in the process supporting the in green highlighted steps.

With regard to the to-be process, several potentials were identified in the baseline interviews with the technology management department (boxes highlighted in orange). This was achieved, using the same, already mentioned set of drivers. In preliminary design, within the adaptive design, the drafting of the ascent assembly sketches could be automated and optimized. Within the detailed design phase, the repetitive tasks of adapting interface components and selecting the gratings bear automation potential. The potentials brought up in the interview study with the engineers are discussed in sections 5.1 and following.

4.2 Automation roadmap

In the baseline interviews, based on the comparison of as-is and to-be processes and the together defined automation roadmap, it was decided to prioritize the motivational driver “Generation of new design alternatives” by exploring the potential “Draw the ascent assemblies paths sketches”. Thereto, it was further decided to develop prototypical solutions for defining the layout/routing of the ascent assemblies, i.e. finding a path network connecting the access points. Their value will be evaluated by RQ3 as listed in Section 5.1. First solutions were recently prototypically realized, as elaborated by Zăvoianu et al. (2019), Hellwig et al. (2019), Zăvoianu et al. (2018) and summarized in Section 4.3.2. Besides improving the adaptive design process through this optimization, integrating the sketching functionality with ACC-Design bears the potential to further automate both the adaptive and original design processes (as elaborated in Section 4.4).

Building on these results and prior to any new automation/optimization deployment, the DA roadmap and a migration plan were validated, as illustrated in Figure 3(b): The first plateau corresponds to the initial as-is process without any implemented and deployed DA solutions. The second plateau corresponds to the current as-is process which is supported by ACC-Design. Finally, the third plateau represents the targeted to-be process in which ACC-Design is integrated with the path layout optimization algorithms. For the second plateau, the anticipated objectives for improving the initial as-is have been identified to be standardization, error rate reduction and lead-time reduction (purple boxes); for the third plateau these objectives are lead-time reduction, cost reduction and generation of novel designs, supporting the out of the box thinking.

4.3 Developed configuration and optimization solutions

The developed applications to automate and optimize initial and current as-is processes, respectively, are presented in the following two subsections.

4.3.1 Automatic crane component (ACC) design.

The product configurator ACC-Design was developed to automate the cumbersome, time-consuming and error-prone detail design. Its implementation has been stepwise refined and extended, and is in operational use for a couple of years. Details can be found in Frank et al. (2014).

The application takes as input a set of standardized parts, a rule base for assembling these parts, and user input to define the particularities of the ascent assembly at hand (e.g. shapes and dimensions of platforms). Furthermore, the engineer defines how the assembly components (i.e. platforms, ladders and stair cases) are combined to obtain the complete ascent assembly. An inference engine processes this input to first represent it in a tree-based standard format before the CAD communication and generation modules send the information to the CAD system to generate the 3D-CAD model and the production drawings. Additionally, the bills of materials and the costs are inferred. In case the engineer needs to adapt the model, the inference engine includes the functionality to change and update the model based on modified user input.

4.3.2 Optimization of routing.

Using ACC-Design, an engineer so far has to manually define how to dimension and combine platforms, ladders and stairs to form an ascent assembly. In this section, attempts to automate and optimize this task are presented. The implementation of these algorithms is on a prototypical level; details can be found in Zăvoianu et al. (2019), Hellwig et al. (2019), Zăvoianu et al. (2018).

In essence, it is assumed that a crane surface can be represented by a cuboid, which is unrolled to a 2D-plane, as shown in Figure 4(a-c) for a gantry of a mobile harbor crane. While this is a simplifying and rather restrictive assumption, the resulting algorithmic problem, namely, the Steiner-Tree Problem (STP) (Gilbert and Pollak, 1968), is NP-hard (Garey et al., 1977). The STP consists of finding the shortest path among a set of predefined terminal nodes (in our case the access points), with the option of freely placing additional nodes (the so called Steiner points) to reduce the length of the overall path. For NP-hard problems, applying genetic algorithms for searching the design space is a viable way for finding optimal design solutions. This approach is taken in all three mentioned papers.

For brevity, we focus on the algorithm presented by Zăvoianu et al. (2019, 2018). The cost function of the optimization is to minimize the Euclidean distance of the path with optional penalties for violating certain angle restrictions (e.g. penalizing angles other than 0^∘, 45^∘ and 90^∘ to achieve designs realizable with standard components) as well as for crossing obstacles. Instead of the Euclidean distance, any other measure of costs, e.g. in form of monetary costs, could be considered. Two solutions are shown in Figure 4(d): the left image shows the shortest path without any angle-restrictions, resulting in partly very steep inclination angles, i.e. very steep, non-manufacturable stair cases; the right image shows the solution for imposing angle constraints of 0^∘, 45^∘ and 90^∘.

The optimized ascent assembly may be restricted to be fully connected, or be divided into several disjoint components, in which case the algorithm chooses automatically the optimal point to split the ascent assembly (to minimize the costs). Two examples are shown in Figure 4(e).

Finally, the inclusion of a second, conflicting objective function was realized, to be able to trade-off solutions, e.g. with different angle restrictions along the so called Pareto-front, as shown in Figure 4(f) for solutions with no angle restrictions and solutions with imposed restrictions of 0^∘ and 90^∘.

4.4 Towards a seamless integrated design automation workflow

The integration of the two applications introduced in Section 4.3, as illustrated in the 5 steps of Figure 5(a), would further automate the ascent assembly design process towards improved leverage of the following potentials: Reduce lead time, reduce costs, and discover and realize new designs. This line of development extends the potential of automatically drawing the ascent assemblies paths sketches. It was performed based on the initial optimization prototype developments and before the industrial evaluation presented in the next Section, towards also evaluating RQ6 as listed in Section 5.1.

The current status of the 5 steps regarding implementation are indicated in Figure 5(a) by the green and purple boxes: Step 1 and 3 are on an idea level with feasibility checked, for Step 2 and 4 a prototypical implementation exists (i.e. step 1-4 are prior to implementation according to the approach of Section 3), and the software of Step 5 is in operational use (i.e. post implementation).

In the first step, starting out from the CAD model of the crane or gantry, the dimensions, corners as well as access points and obstacles have to be defined and translated to the 2D representation. Currently, the representation of the 2D abstraction is generated manually. For semi-automatic generation, a user interface could be developed in which the engineer enters the required data (access points, crane corners, obstacles), and the translation is done automatically. Alternatively, the engineer specifies the points directly in the CAD model, which are then automatically extracted to generate the 2D representation. This representation is stored in a pre-defined standard format, which serves as input file for the optimization algorithms.

Secondly, any of the presented optimization algorithms of Section 4.3.2 is applied to the 2D representation to infer a path between the access points while avoiding obstacles. The algorithm outputs another file containing the network representing the assemblies. Alternatively, the design engineer could draw the routing manually on the 2D plane.

In the third step, the solution (s) of the optimization algorithm are presented to the design engineers who can verify and post-process the solution (s) according to their standards. Post-processing options for the deployment include moving, adding and deleting access points and obstacles, pulling the lines of a solution of the output of the algorithm, interact with the optimizer in terms of modifying the solution and re-starting the optimization, as well as running a multi-objective optimization and selecting from several results. The post-processed solution network is stored into a file, which is used as input for translating the lines to ACC-Design.

In the fourth step, the post-processed network is analyzed and translated to an XML-file, which contains assembly and assembly combination information in the format specified for ACC-Design input files. During the translation process, restrictions imposed by ACC-Design, e.g. that certain combinations are not possible (such as directly connecting a ladder to a stair case without a platform in between), are handled.

Finally, in step five, the generated XML-files are opened in ACC-Design. The engineers can now verify the input and adapt settings not related to the dimensions, such as the safety cage of the ladders or the stability of platforms (e.g. standard or extra strong), and generate the ascent assembly draft.

5. Industrial evaluation

An interview study was conducted with six engineers, distributed evenly on the two departments for designing ship- and offshore cranes (original design) and mobile harbor cranes (adaptive design). The interview study included an assessment of benefits/potentials and limitations of ACC-Design after implementation and of the optimization and integrated workflow before implementation, as well as questions about acceptance and usage of the integrated workflow, and DA solutions in general.

Towards this end, the integrated workflow procedure was explained to the interviewees on a conceptual level with a figure similar to Figure 5(a), the optimization using figures similar to the ones in Figure 4, and the interaction/post-processing within Step 3 as a list of options with explanations. For Step 4, the translator was first explained on a conceptual level, and after an initial judgment, the prototypical implementation was shown to validate the effect of a prototype demonstration. Finally, the complete workflow was demonstrated by showing the engineers the expert solution in the abstract representation and the generated 3D-CAD model inferred by the prototype translator as shown in Figure 5(b), as well as several solutions of the optimization algorithm with two entry points and varying angle restrictions, one of which is shown in Figure 5(c).

5.1 Research questions

The following research questions (RQ) evaluated in the following are derived and have been made explicit at different passages beforehand. They focus on early-stage DA support and seamlessness.

RQ1.

(Early-stage DA uncertainties): Will it again appear that there are uncertainties with respect to the awareness of available opportunities, recognition of potential of applying DA and ability to define the automation task (Rigger et al., 2016)?

RQ2.

(Early-stage DA value): What is the value of early-stage DA in this context?

RQ3.

(Prototypes value): What is the value of demonstrating new functionality with prototypes?

RQ4.

(Optimization suitability): Will the path layout optimization be more suitable to early-stage original design or early-stage adaptive design in this context?

RQ5.

(Control vs full automation): Do designers value more control and decision-making vs fully automated design tasks, or vice versa?

RQ6.

(Seamlessness): What aspects of seamless integration of DA tools do designers value?

5.2 Potentials/benefits and limitations

For the qualitative validation of the potentials/benefits and limitations of the integrated workflow and its components the interviewees could, on the one hand, pick pre-defined answers from a list of drivers for design automation and optimization deducted from an interview study among about 50 manufacturing companies (Rigger and Vosgien, 2018). On the other hand, to gain deeper insights and to double check the answers, open questions regarding the benefits and potentials were posed. These answers were structured according to the list of drivers, and, to fully reflect the participants’ answers, the list was enriched with additional criteria (in particular enhanced change management, in terms of detecting required changes and propagating them, as well as reproducibility). While Steps 1 to 4 of the integrated workflow were assessed qualitatively before implementation (corresponding to Step III of the introduced approach), Step 5 was largely validated qualitatively after implementation (corresponding to Step VI.II), but also a limited quantitative success validation (corresponding to Step IV.I) on time savings was performed by extracting lead times from the ERP system. The results are summarized in Figure 6. They are all based on the already mentioned same set of motivational drivers.

Overall, with regard to the expected benefits of the complete workflow, the quick solution generation, which enables the development of customer specific designs, would become even more easy by connecting ACC-Design to the optimization. In Steps 1 and 3, change management is supported by moving, adding and deleting access points in the abstract representation, which also helps in reacting fast to changes in customer requirements. A straight forward potential of Step 2 is to reduce costs by finding a shorter/cheaper ascent assembly solution. However, the biggest benefit of the optimization algorithm, emphasized by all of the adaptive design interviewees plus one other, lies in the generation of alternative, potentially novel solutions, by using a multi-objective optimization, setting appropriate angle restrictions or varying the number of entry points. Seeing between two and five solutions would help the engineers to faster select a concept because there is less need to search for previous solutions. This also means that the knowledge of how to route the ascent assembly would be stored in the algorithm, i.e. preserved independent of the engineers. With this regard, it was, e.g. positively observed by the interviewees that the solutions with two entry points of the optimization algorithm (Figure 5(c)) inferred the same splitting point of the ascent assembly as the expert solution (Figure 5(b)). While for the experts the reason for the splitting are because of mounting considerations (possibility to mount part of the assembly later), the optimization algorithm looked for the shortest possible solution. All of this is directed at answering RQ2. Potentials identified in Step 3 are particularly addressing RQ5: the potential of enhancing change management and faster reacting to changes of customer requirements, as discussed together with Step 1, is further enabled by post-processing a solution (e.g. pulling lines), which helps in correcting and updating a solution. For example, the ascent assembly inferred by the algorithm without any angle restrictions (left image of Figure 4(e)) could not be directly generated in ACC-Design because too short and too steep stair cases were contained in the network. Establishing a knowledge base could also be part of the post-processing, e.g. by only allowing certain changes when pulling the lines. The translation of the network to ACC-Design in Step 4 was perceived as very positive by all interviewed engineers. They see the potentials to save time and reduce errors, as it automates a rather cumbersome task. These two potentials also yield a cost saving potential as well as the possibility to react faster to changes of customer requirements. This relates to RQ6.

Finally, the evaluation of ACC-Design posterior to implementation revealed a rather long list of benefits, including all drivers initially listed in the automation roadmap. For the implementation of ACC-Design, the parts and their assembly procedures were standardized. Thereby, the designs’ consistency was improved, costs reduced (fewer parts need to be managed) and error rates decreased (especially through the incorporation of company and industrial norms and standards as well as the automatic selection of structurally appropriate components; these points were mentioned as large benefits by all engineers). The automation further resulted in the establishment of a knowledge base, enabled the re-use of this knowledge and lead to reproducibility of the results, as the design procedure, norms/standards and structural calculations are stored and applied to every new design, which helps in understanding and trusting the solution. Finally, lead-time reduction (reduction of time spent for repetitive tasks - time savings) is especially achieved through the following three points:

1. As highlighted by several engineers, automatically obtaining a draft and detail design including the CAD model and production drawings (i.e. visualizing designs and generating documentation) eliminates the time-consuming search for suitable parts.

2. The ability of updating the design using the update-functionality of ACC-Design enhances change management allowing to correct inaccuracies and faster react to changes in customer requirements.

3. Integrating norms and structural requirements in the product configurator supports the norm/standard checking tasks and replaces time-intensive calculations.

For the quantitative success validation, as mentioned in Frank et al. (2014), up to 90 per cent of the overall design time can be saved by using ACC-Design. For a more detailed analysis, lead times were extracted from the ERP-system revealing that an average overall reduction of about 50 per cent is achieved, with roughly 30 per cent of time being saved in the early design phase and almost 60 per cent in the detail design phase. As stated by several engineers, the saved time can be used to improve the solution quality, offer several solutions and hence better meet the customer requirements.

For the drawbacks of the overall workflow, the current application area seems to be rather restrictive (require crane surface which can be unrolled), and only minor time and cost savings are expected when considering the complete workflow. This of course questions the ROI of such an implementation. One major reason for these limitations is the 2D representation (Steps 1 and 2). All engineers agreed that for an actual implementation, the representation would need to be in 3D and automatically generated from the CAD model (such that changes in the CAD model are propagated to the abstract representation). Here, the idea level with only the feasibility checked produced an artefact to be overcome. With regard to the optimization algorithm in Step 2, some further limitations were seen, related to RQ2. The main point of criticism was the extensive use of ladders: In practice, there is an important distinction between main ascent assembly components (used almost daily) and side ascent assembly components (used at most a few times per year for maintenance), where the former should consist of stairs and platforms only, and the latter may well additionally contain ladders. This distinction is currently not made in the algorithm. Furthermore, access points are not always strictly given as exact points, rather they are picked from a certain access area. Often, solutions between different cranes/gantries should look alike (e.g. when several cranes/gantries are installed next to each other), meaning that the optimization, even though it could find cheaper results for some of the cranes/gantries, is not required. Also for the post-processing in Step 3, some restrictions should be enforced, such as the distinction between main and side ascent assemblies (addressing RQ6). Solely for Step 4, i.e. RQ6, no limitations were mentioned.

With regard to the limitations of ACC-Design in Step 5, for certain specific use-cases, ACC-Design cannot handle the required exceptions (e.g. forcing a stay at a specific position, as these are placed in an optimally spaced distance) and manual fine tuning is required. For quickly drafting a solution it had been good to exclude all small parts (e.g. screws) when generating the CAD model. However, for the detail design, exactly this feature is required. ACC-Design also does not feature a functionality to search for already generated assemblies (in previous projects), which would further assist in re-using not only parts but also assemblies. Finally, as highlighted in Figure 3 (a) in the detail design process, the red boxes also bear potential for improvement and could be included as additional functionalities in ACC-Design. On the one hand, weld-on plates for attaching the ascent assembly to the crane have to be manually adapted (because the distance information between crane and ascent assembly is not known in ACC-Design) and other weld-on and bolted parts for e.g. attaching cables and lights have to be added manually. Especially with regard to the last group of parts, adaptations made via the update function of ACC-Design do not apply to these parts. On the other hand, after finishing the drawings, gratings have to be added to the platforms, which includes a manual search for similar parts; this is also a cumbersome task not covered by ACC-Design and a candidate for automation.

5.3 Acceptance and usage

Regarding the integrated workflow, as Steps 1-4 are not yet in operational use, an evaluation focusing on potential usage and acceptance has been conducted. The validation of the targeted integrated automation workflow is divided into four parts: overall evaluation, optimization, interaction, and seamlessness. The interviewees were posed several questions, which they were asked to rate according to a five-tier scale and comment on their ratings. The questions and answers are shown in Table I (top table). The results for original design (ship- and offshore cranes) and adaptive design (mobile harbor cranes/gantries) differed for certain questions, and are hence shown separately.

With regard to the overall evaluation of the workflow, the interviewed engineers were mostly positive, seeing the sense of such a workflow integration, believing that it could work and being open to use it, naturally under the condition that the solutions are meaningful. For use cases with little degrees of freedom (such as in the original design) the application seemed less realistic to be useful, but for other application areas, also these engineers stated that they can well imagine that it supports the work, addressing RQ4.

With regard to the optimization, the current intention to use such a tool is not very high, the reasons for which are three-fold (addressing RQ2):

1. For the original design, the concept of routing the ascent assembly has to be defined simultaneously to design the crane itself because of strict space restrictions leaving almost no degrees of freedom to choose a routing.

2. For the adaptive design, even though there are significant degrees of freedom, a concept is defined rather quickly, and hence, there is no immediate time gain when using an algorithm.

3. In general, there are missing restrictions in the algorithmic implementation, such as the distinction between main and side ascent assemblies.

However, defining a routing manually on an abstract representation is perceived as a useful way of making drafting easier in both original and adaptive design. Regarding the trust in an optimization solution, the interviewees responded that it grows with seeing useful solutions. The engineer who responded to rather not use the solution could not imagine that it is accurate enough to fulfill all the required space restrictions. Lastly, the run-time of the optimization was also of different importance, ranging from instant to “half an hour or more, if the solution is good”.

Addressing RQ5, the importance of interaction possibilities was rated very high for actually using such an application, especially for post-processing the solution network (pulling lines), but also in terms of moving access points and obstacles to be able to react to changes in the surroundings. An important aspect was the connection of the CAD-model of the crane/gantry to the abstract representation, such that any changes (e.g. moving the cable reel) are propagated automatically (RQ6, consistency, change management). The interaction with the optimizer itself was less popular, and also only under the restriction, that made changes (e.g. pulling a line in a certain place) are not adapted anymore by the optimizer.

Addressing RQ3, 4 and 5, the selection from several solutions was judged very useful, especially in terms of trading-off solutions only containing stair cases and solutions also allowing ladders. Here, consistently to the explorative identification of potentials as to Step 2, the generation of alternative solutions is evaluated most useful.

Finally, with regard to the seamlessness, i.e. the integration of the optimization and ACC-Design, the usefulness especially of the translator of Step 4 was recognized and acknowledged (again addressing RQ6). For the same reason, the workflow was judged to increase the value of ACC-Design. As stated above, before the first judgment, the translator was only presented on a conceptual level, to evaluate the effect of showing a working prototype. The answers to the above questions did not change after seeing the prototype. However, for most engineers, the demonstration of the prototypes left a positive impression, making the idea more concrete and showing its applicability. Furthermore, running prototypes raised trust and, although not evaluated, seemed to improve recall.

To obtain more general statements about the acceptance and usage of automation/optimization applications, we asked the engineers to rate certain criteria from a general point of view, and comment on their judgment. The results are shown in Table I (bottom table).

The two most important factors for accepting a design automation or optimization application are the practical relevance (i.e. appropriateness of the tool for the task at hand) and the possibility to influence the solution finding (again answering RQ5 in the same direction as before), closely followed by trusting and understanding the solution. While a stepwise introduction and seeing/testing the application beforehand with a prototypical implementation does not seem to be a crucial aspect for the acceptance, the stepwise integration of the users in its development is. Here, several of the interviewees pointed out that key-users should be integrated in the development phase, especially for determining the required input and the expected output, and with a growing integration the further the development is.

When asked about various aspects of a DA application, usability and functionality were rated the highest. With regard to functionality, an engineer stated that first of all frequently performed tasks with a wide user circle should be automated. They also appreciate small tools, which facilitate or support their daily work, and prefer add-ins for the used program (e.g. the CAD-system) over stand-alone applications, as this facilitates the usage. This explains the seemingly surprising order of ratings for functionality and usability. This again answers RQ5 consistently. The importance of comprehensibility/traceability ranged from very important to neutral. While for some engineers it was most important that the resulting solution looks reasonable, others required to understand the w2ay the solution was found especially when starting to use a tool. Seamlessness was also judged to be of varying importance: Lower ratings were commented with the usefulness of small supporting tools facilitating daily work (emphasizing the control aspect of RQ5), and higher ratings with the importance of connecting DA application to the CAD- and PDM-system (emphasizing automatic synchronization of data as to RQ6), tremendously simplifying work and helping in standardization.

5.4 Study validation

According to (Robinsons, 2016; Venkatesh et al., 2013), there are two primary validation issues, roughly analogous for quantitative and qualitative evaluation, i.e. reliability and validity of measures.

As to the approach adopted, it is based on standard workflow modeling in the context of process re-engineering/improvement. As-is processes have all been consistently produced. The potentials/benefits and drawbacks have all been evaluated based on the same set of motivational drivers as explained before (although with some extra extensions from open questions), where qualitative evaluation was first performed for exploratory purposes, yielding consistency in Figure 6. Afterwards, for more detail, the quantitative results in Table I have been acquired. Both Figure 6 and Table I have been given to the study participants for double checking after the study and before submission of this paper. Although this doesn’t ensure exact measurement reliability, it works towards consistency. During all the interviews, the values displayed in Table I remained the same before and after prototype demonstrations as compared to conceptual explanations of the 5-steps workflow.

While external validity of the answers of the research questions is already framed by the text passages they are derived from, further validation is as follows:

RQ1.

(Early-stage DA uncertainties): Indeed, there are again uncertainties found with respect to early-stage DA with a major exception of “generation of alternative solutions” in Figure 6 and “selecting from several solutions” (in particular for adaptive preliminary design) in Table I. The latter consistently achieved maximum rates.

RQ2.

(Early-stage DA value): Overall, the highest value of early-stage DA is in what is emphasized in RQ1. Thus, the validity argument is the same as for RQ1.

RQ3.

(Prototypes value): Although for both results sets (Figure 6 and Table I), the answers didn’t change in reaction to prototype demonstration, we argue that without prototype implementation, neither the optimization results could have been presented in such detail nor the integration in the five-step procedure would have emerged so concretely. Nevertheless, the internal validity of this statement cannot be proved high.

RQ4.

(Optimization suitability): The path layout optimization is more suitable to early-stage adaptive design in this context. This is demonstrated with very high internal validity by all the relevant average values of the top table of Table I consistently showing higher ratings for adaptive design than for original design.

RQ5.

(Control vs full automation): Designers value more control and decision-making vs fully automated design tasks in the given context. Overall, this is a trade-off relationship. In this study, however, as can be seen in the interaction segment of the top table of Table I, it can be validated that it is control preferred over full automation. In particular, interaction with the optimizer is rated lower than choosing from its results. This is further supported by the answers regarding clear scoping of often smaller DA tools over full/seamless process automation.

RQ6.

(Seamlessness): The dominant aspect to the participants of the study is the translator from 2D to 3D. Such translators are commonly used in all kinds of model-based design approaches. However, seamlessness doesn’t appear to be a value in its own, but only supportive.

The validity of the answers to RQ1 to 6 are further substantiated by the discussion and conclusion, which have both been evaluated as correct by the industrial study participants.

6. Discussion

The results of the industrial evaluation and discussion are based on the opinions of the interview participants; any generalization would require further investigations and validations. The interviews were conducted by the developers of the integrated workflow, which allowed the interviewees to ask questions in case of lack of clarity, but could have distorted the assessment due to the missing anonymity.

Assessing in-use tools after implementation cannot only be used to evaluate these tools (in our case ACC-Design, Step 5 of the integrated workflow), but also help in gathering ideas for further lines of developments (e.g. automating adjustment of weld-on/bolted parts) because during such a validation limitations as perceived by the end-users are revealed. Similarly, by validating ideas and prototypes before the actual implementation (in our case Steps 1 to 4 of the workflow), important directions of developments can be identified and corrected early on (e.g. option to draw sketches manually; extension to 3D). Towards this end, showing engineers solutions generated by the prototype not only left a positive impression, but especially helped in extracting implicit knowledge about the design of ascent assemblies (e.g. main vs side ascent assemblies, access points vs access areas).

The rather critical validation of the optimization itself can be mostly reduced to three reasons. (1) The use cases were on extreme ends of the application range: While solving the routing problem for gantries is a rather easy task, for ship- and offshore cranes, it is intertwined with the crane design itself, leaving little to no room for optimization. Thus, time and cost saving potentials were also judged to be minor. The engineers, however, acknowledged that a similar algorithm could be useful for routing pipes, tubes, ropes or cables, where the problem solving is more involved. (2) The 2D representation, on which the algorithm operates, is in practice simply too restrictive (not enough application areas) and unconventional (engineers mainly work in 3D), even if it is from a theoretical point of view very interesting and challenging to solve. Thus, for a proof-of-concept, the 2D representation is a valid starting point, and the algorithms are implemented in a way such that a generalization to 3D is rather straightforward, but could come with increased runtime. (3) Not all practically relevant restrictions are incorporated in the algorithms (e.g. main vs side ascent assembly, access points vs access areas), leading to unfeasible results.

On the other hand, the generation of several solutions using the optimization was highly appreciated among the participants, as it supports ideation of problem solving. This is contrary to the answers given to the acceptance and usage assessment from a general perspective, where engineers were still most interested in automating repetitive tasks, and also contrary to the results of the study in Rigger and Vosgien (2018), where obtaining alternative solutions was ranked as a minor driver for implementing DA. This discrepancy can be explained by the fact that only regarding the former part the engineers have seen an approach for finding several solutions before answering the questions. Thus, engineers are open to such approaches, however, do not seem to see the potentials themselves. Therefore, well scoped prototypes seem an essential ingredient of potential evaluation. This is in line with observations of the literature, where it is stated that DA practitioners are often not aware of the available opportunities (Bolognini et al., 2012), do not recognize the potential of applying DA (Verhagen et al., 2015) and lack the ability to select and implement appropriate methods once the automation task has been defined (Amen et al., 1999).

The answers of the assessment prior to implementation point in the directions that engineers prefer applications where the design task itself and the final decision-making are left to the engineers. This is supported by three statements:

1. The option of replacing the optimization result with a manually drawn sketch was perceived as very useful, especially when connected to the automatic translation of the network to ACC-Design, also facilitating loops between the abstract and concrete representation.

2. Allowing the designer to choose from several solutions was highly appreciated.

3. Influencing and post-processing the solution within the workflow was of great importance.

These aspects allow to overrule the application’s solution, making the engineers feel useful and needed. Furthermore, it helps in ensuring that the final solution is not blindly accepted and that the engineer, potentially after adapting it, checks and confirms its feasibility. Thus, the integrated workflow is appreciated for being a supporting tool rather than a tool automating the complete design task, which is in accordance to the statement of (Dym and Brown, 2012) that stepwise automation of well-defined parts is often preferred over full automation.

Based on the results and their discussion, the decision-makers obtain a profound basis for determining further development steps and directions. This process is currently on-going.

The findings reported in Section 5 and discussed above have proven the applicability of the approach introduced in Section 3, even though it was not entirely applied to one single DA application and only with a minor quantitative assessment. Through the systematically derived assessment criteria of Rigger and Vosgien (2018) and their case-specific enhancement, not only the suitability and effectiveness of the solution were addressed, but also acceptance and usage aspects. While such a generic set of assessment criteria for DA tasks guides and supports the validation steps and enhances comparability of methods when searching for appropriate DA methods for a given DA task, these criteria also need to be refined specifically for the addressed problem and the chosen solution.

The used approach is a starting point for developing a generally applicable, systematic methodology. Regarding Steps I and II and the identification of measurements in practice, a more comprehensive systematic that not only considers design processes as to a task precedence model (Wynn, 2017), but also takes into account the supporting tools and technologies needs to be developed. This permits definition of design performance assessment also from a software quality point of view such as usability.

In this context, first steps have been taken to appropriating long standing traditions in usability engineering to a DA tool context. Role models were heuristic evaluation and discount usability engineering, showing in a scientifically substantiated way good coverage with only five participants, in particular as to exploratory studies (Nielsen, 1994).

7. Conclusion

The presented case study for ascent assembly design was used to demonstrate the approach for validating previous and future developments, i.e. assessing the success/benefits and potentials, respectively. The industrial evaluation after implementation of the KBE-application ACC-Design (partly based on lead times with and without ACC-Design as documented in the ERP of LWN) and before the implementation of the routing optimization and the seamless workflow integration of the two applications were conducted by interviewing engineers. Even though the complete approach introduced in Section 3 was not applied to one single DA solution, the case study and the results of the interviews clearly demonstrate the applicability of the approach, in particular by structurally assessing the (expected) added value of different kinds of DA solutions.

The results of the evaluation of ACC-Design revealed that not only the anticipated benefits (standardization, error and lead time reduction, as marked in Figure 3(b)) were met, but additionally a couple more, not immediately sought for benefits were achieved. On the other side, for the integrated DA workflow connecting the optimization with ACC-Design, the evaluation prior to implementation points in the direction that not all anticipated benefits are realistic. However, the possibility to generate several solutions was greatly appreciated by the engineers.

This last point closely links to the central motivation of the paper - increasing the awareness, understanding and adoption of DA applications in early design stages. While the interviewees themselves see most improvement potential in repetitive/routine design tasks, once the idea of obtaining several solutions early in the design process was prototypically demonstrated, it was very appealing, especially when connected to the existing, well-known product configurator. Thus, by using prototypes, the awareness of what is possible is raised, and with it the acceptance of such solutions.

Based on the above observations, the results of the interview study as well as the discussion, there are several lines of future work:

• The restrictive 2D representation has to be extended to 3D for operational use, and with it the automatic generation of this representation from the CAD model should be realized.

• There are several extension that would enhance the practical relevance of the optimization algorithms: When using a 3D representation, also the algorithm needs to be adapted to 3D, which is rather straightforward because of the flexible implementation. Further restrictions should be incorporated in the algorithm, such as the distinction between main and side ascent assemblies (e.g. by penalizing unwanted connections such as ladders along certain paths) or allowing access points to be placed within an interval (e.g. heuristically by running the current algorithm in parallel with different placements). Finally, for post-processed solutions, the algorithm should include the option to not alter certain paths, e.g. by setting the costs for these paths to zero.

• In the presented workflow, an option could be introduced to choose between running the optimization and manually drawing the network sketch. For manual sketching, instead of generating a 2D/3D representation, the routing could be defined directly in the CAD model and translated to ACC-Design.

• Further automation potential was identified for some routine tasks in the detail design following the presented workflow, in particular for defining and re-fining welding and mounting components, as well as searching for suitable gratings for the platforms. These are typical routine tasks that could be automated or at least computationally supported.

• Finally, the presented approach should be extended and generalized to serve as a general methodology for design automation potential identification and validation using both qualitative and quantitative assessments before and after implementation. Towards this end, DA templates and metrics have to be further developed and integrated into the methodology, and the entire method has to be applied to a single case study.

Figures

Figure 1.

Examples of (a) an offshore crane and (b) a gantry of a mobile harbor crane with ascent assemblies highlighted in red

Figure 2.

Approach for DA potential identification and assessment prior to implementation (Block A) and for posterior DA success validation (Block B)

Figure 3.

Method application in (a) the as-is process with deployed product configurator and (b) the automation roadmap and migration plan

Figure 4.

(a) A gantry of a mobile harbor crane; (b) 3D representation of the gantry by stacking two cuboids on top of each other with blue dots depicting access points and red rectangles obstacle areas; (c) unfolded 2D representation of the gantry with grey areas representing non-accessible spaces, which are treated as obstacles; (d) solutions with one entry point and different angle restrictions: left – no restriction; right – 0^∘, 45^∘ and 90^∘; (e) solutions with two entry point and different angle restrictions: left – no restriction; right – 0^∘, 45^∘ and 90^∘; (f) multi-objective optimization results when trading-off geometrical/Euclidean optimality and domain-specific optimality (featuring only 0^∘ and 90^∘ angles)

Figure 5.

Sub-figure (a) shows the integrated automation workflow of ascent assembly design, (b) the abstract representation and generated ascent assembly of the expert solution using the workflow and (c) an optimized solution with angle restrictions of 0^∘, 45^∘ and 90^∘ and the generated ascent assembly (only larger component of the CAD-model is shown)

Figure 6.

Potentials/benefits (in blue) and limitations (in orange) of the integrated workflow

Table I.

Top table: overall workflow evaluation, separately for the original (“orig.”) and adaptive (“adapt.”) design tasks including average ratings (“Av.”). Bottom table: General evaluation of acceptance and important aspects for design automation and optimization applications. Scale: –2 (absolutely no), –1 (rather no), 0 (neutral), +1 (rather yes), +2 (absolutely yes)

Question	Design	Evaluation						Av.
Overall evaluation
Does it make sense?	Orig.	1	0		1			0.67
	Adapt.	1	1		2			1.33
Do you think it works?	Orig.	1	1		1			1.00
	Adapt.	1	2		1			1.33
Would you use it?	Orig.	1	0		1			0.67
	Adapt.	1	2		1			1.33
Optimization
Would you use it?	Orig.	–1	–1		0			–0.67
	Adapt.	1	0		1			0.67
Would you rather draw the path	Orig.	2	2		1			1.67
manually?	Adapt.	2	1		1			1.33
Would you trust it?	Orig.	1	1		1			1.00
	Adapt.	1	1		1			1.00
Would you use the proposed solutions?	Orig.	1	–1		1			0.33
	Adapt.	1	2		1			1.33
Does run-time need to be instant?	Orig.	2	1		1			1.33
	Adapt.	2	–1		0			0.33
Interaction
In general?	Orig.	2	1		2			1.67
	Adapt.	2	2		2			2.00
For moving access points and obstacles?	Orig.	1	1		1			1.00
	Adapt.	1	2		2			1.67
Pulling lines?	Orig.	2	2		2			2.00
	Adapt.	2	2		2			2.00
With the optimizer?	Orig.	0	1		1			0.67
	Adapt.	1	2		1			1.33
Selecting from several solutions?	Orig.	2	1		1			1.33
	Adapt.	2	2		2			2.00
Seamlessness
Is the integration of optimization and	Orig.	1	1		1			1.00
ACC-Design important?	Adapt.	1	2		2			1.67
Does this workflow increase the value of	Orig.	2	1		1			1.33
ACC-Design?	Adapt.	2	1		1			1.33
Question		Evaluation						Av.
Acceptance
Practical relevance		2	2	2	2	1	2	1.83
Stepwise introduction		1	0	–1	0	0	0	0.00
Stepwise integration of users		1	2	1	1	1	1	1.17
See/test prototype		1	0	–1	0	1	0	0.17
Trust/understand solution		1	1	2	2	2	1	1.50
Influence solution finding		2	2	2	2	1	2	1.83
Aspects
Functionality		2	1	1	1	1	2	1.33
Usability		2	1	1	1	2	2	1.50
Comprehensibility/traceability		0	0	2	1	1	1	0.83
Seamlessness		2	0	0	0	1	1	0.67