Augmenting the space domain awareness ground architecture via decision analysis and multi-objective optimization

Abstract

Purpose

The US Government is challenged to maintain pace as the world’s de facto provider of space object cataloging data. Augmenting capabilities with nontraditional sensors present an expeditious and low-cost improvement. However, the large tradespace and unexplored system of systems performance requirements pose a challenge to successful capitalization. This paper aims to better define and assess the utility of augmentation via a multi-disiplinary study.

Design/methodology/approach

Hypothetical telescope architectures are modeled and simulated on two separate days, then evaluated against performance measures and constraints using multi-objective optimization in a heuristic algorithm. Decision analysis and Pareto optimality identifies a set of high-performing architectures while preserving decision-maker design flexibility.

Findings

Capacity, coverage and maximum time unobserved are recommended as key performance measures. A total of 187 out of 10¹⁷ architectures were identified as top performers. A total of 29% of the sensors considered are found in over 80% of the top architectures. Additional considerations further reduce the tradespace to 19 best choices which collect an average of 49–51 observations per space object with a 595–630 min average maximum time unobserved, providing redundant coverage of the Geosynchronous Orbit belt. This represents a three-fold increase in capacity and coverage and a 2 h (16%) decrease in the maximum time unobserved compared to the baseline government-only architecture as-modeled.

Originality/value

This study validates the utility of an augmented network concept using a physics-based model and modern analytical techniques. It objectively responds to policy mandating cataloging improvements without relying solely on expert-derived point solutions.

Citation

Vasso, A., Cobb, R., Colombi, J., Little, B. and Meyer, D. (2021), "Augmenting the space domain awareness ground architecture via decision analysis and multi-objective optimization", Journal of Defense Analytics and Logistics, Vol. 5 No. 1, pp. 77-94. https://doi.org/10.1108/JDAL-11-2020-0023

Introduction

Through a worldwide network of ground-based telescopes and radars as well as space-based optical platforms, the US Government (USG) collects and processes data on over 23,000 satellites, rocket bodies and pieces of debris with a diameter of 10 cm or greater in orbit (Lal et al., 2018). These Resident Space Objects (RSOs) are present in both the Near Earth and Deep Space orbital regimes. Deep Space RSOs are categorized as those with periods greater than 225 min and include RSOs in Geosynchronous Orbit (GEO) (USSTRATCOM, 2004a).

The general knowledge gleaned from collection of objects is referred to as Space Domain Awareness (SDA), which consists of both cataloging and characterization of RSOs. Cataloging includes identifying particular RSOs and projecting their future locations using astrodynamics equations; characterization ascribes properties, behavior and intent to the RSO. A second, related mission is Space Traffic Management (STM), which involves communicating conjunction probabilities to RSO owners and operators to prevent collisions and adding additional debris to congested operating areas.

For several historical reasons, the USG has become the world’s de facto provider of no-cost, basic RSO cataloging and STM information. Multiple users rely on this data to ensure their defense, civil and commercial missions remain executable. Any disruption or inability to provide timely, rigorous information increases the risk of loss of mission. Disruptions in any space-based service – communications, timing, navigation, imagery collection – will impact users worldwide and imposes ripple effects to multiple users. Additionally, repeated failures to modernize equipment coupled with the projected increase in future space services and a shifting focus on new threats seriously jeopardizes the USG’s ability to maintain space superiority and ensure safe STM for the world. National-level policy outlined in Space Policy Directive 3 (SPD-3) now mandates improving this situation.

Fortunately, nontraditional sensor providers have matured to the point where their contributions are routine, reliable and prime for incorporation into USG SDA processes. These providers are defined as one-off civil and scientific sensors and well-developed worldwide commercial tracking networks. The USG has recognized the utility of these capabilities as well as outlined how to vet these providers for future incorporation. However, no framework exists to assess the future system of systems of USG/nontraditional sensors, herein referred to as the Augmented Network (AN), nor how to approach designing the AN in an objective, optimal manner.

This study assumes a notional future scenario in which a USG Systems Program Office (SPO) is charged with designing the AN. The SPO will incorporate Deep Space cataloging data from one civil, one scientific and a set of sensors from three distinct commercial companies proposing a total of 56 sensors. Civil and scientific data from two large telescopes is assumed to be contributed for free without any ability to influence taskings. The SPO is assumed to be allocated $25m to acquire some portion of commercial capabilities and decides to purchase exclusive rights to fully task and incorporate cataloging data from small-aperture, Narrow Field of View (NFOV) telescopes.

The challenge becomes determining how to objectively select which sensors to include. Comparisons to source documents and other studies prompt three key performance measures be pursued: capacity, amount of time unobserved and coverage. Various AN permutations, or architectures, are modeled and simulated on two different nights and assessed using Multi-Objective Optimization (MOO) to evaluate performance subject to the cost constraint. A heuristic algorithm is used to smartly assess the tradespace of 10¹⁷ possible architectures. Pareto domination is applied post priori to identify the top performing architectures, while additional management-based considerations further reduces the tradespace. The decision-maker is left with a handful of choices culled from the best-performing architectures and higher confidence that a high-performing choice has been selected than by merely relying on a Subject Matter Expert’s proposed point design.

Background

System architecting, decision analysis and optimization

Maier and Rechtin define the discipline of systems architecting as the art and science of creating and building complex systems through use of qualitative heuristic principles and quantitative analytical techniques (Maier and Rechtin, 2010). System needs and requirements permit the architect to define potential architectures, while measures help capture performance (Crawley et al., 2016). The evaluation of alternatives is aided by decision analysis. A common decision analysis approach uses values-focused thinking, which uses decision-maker objectives to quantify the performance of alternatives (Parnell et al., 2013).

Many problems have competing goals, complicating comparisons. Typical comparisons use either a weighted sum or MOO to illuminate the best performers. Crawley et al. recommend using MOO to evaluate a system’s design tradespace, which is defined as “numerous architectures, represented at lower fidelity and evaluated with a few simple key metrics,” and using Pareto and near-Pareto optimality to identify solutions (Crawley et al., 2016). Pareto optimality consists of ranking competing solutions based on their ability to outperform others in all objectives into successive fronts, where solutions on the same front do not outperform each other in all objectives. Rhodes and Ross outline an additional use of Pareto optimization in architecting via Epoch-Era Analysis, in which a system is assessed during multiple periods of fixed needs and contexts throughout its lifespan such that high-performing architectural choices common to all periods are posited as those with greater value (Rhodes and Ross, 2010).

Astrodynamics, scheduling and optical collection

Because of limitations in sensor capabilities, approximations in equations and models and measurement errors, an RSO’s true position and velocity is rarely known. The overall uncertainty of the state is represented by the RSO’s covariance, which grows as time elapses. Maintaining a low covariance on all RSOs by conducting frequent observations is thus paramount to maintaining SDA. Preferably, “observations should be taken at different positions on a satellite’s orbital path […] ideally, cover[ing] the full 360 degrees of an orbit” but this is unrealistic given resource constraints (USSTRATCOM, 2004b). Observations are instead planned using scheduling theory algorithms and analyst judgment.

Near-Earth RSOs are traditionally tracked using ground-based radars, while Deep Space RSOs are monitored using ground-based telescopes. Space-based observation platforms are capable of tracking RSOs in any regime. Optical observations using NFOV telescopes typically use the sidereal collection mode, in which the telescope focuses on the background stars at the suspected location of the RSO and collect light over a short time interval, thus forming a streak in the data from which the endpoints are used to generate two observations (Campbell et al., 2018). Multiple observations on an RSO are ultimately combined in a process called orbit determination in which RSO state and covariance information is updated. This information is used to propagate an RSO’s position forward or backward in time to conduct the SDA and STM missions.

Figure 1 outlines the typical optical collection chain for a ground-based telescope, which are traditionally limited to nighttime collections because of daytime brightness. Ultimately, a sufficient signal-to-noise ratio (SNR) at the telescope’s sensor array is required for a successful collection. Howell’s (2006) and Shell’s (2010) equations are useful in defining the radiometry of the problem. Schmunk’s approach estimates the RSO’s light at the telescope given a known standard visual magnitude (M_v) (Schmunk, 2008). A general background brightness of 19 to 21 M_v/arcsec² may be assumed for dedicated sites on clear moonless nights while Krisciunas and Shaffer’s work informs simulating lunar brightness (Krisciunas and Schaefer, 1991). Tools such as the AFIT-developed Laser Environmental Effects Definition and Reference (LEEDR) routine model atmospheric conditions (Fiorino et al., 2008). Clouds may be modeled stochastically based on known empirical data sets.

Current USG SDA processes, challenges and impetus for change

RSO cataloging is performed by several ground and space-based sensors. Three Ground-Based Electro-Optical Deep Space Surveillance (GEODSS) sites, each with three 1 m telescopes in New Mexico, Maui and Diego Garcia comprise the major Deep Space ground collection systems. An improved future system, the Space Surveillance Telescope (SST), is planned to begin operations from Australia in 2022. Three space-based platforms – Space-Based Space Surveillance (SBSS), Operationally Responsive Space 5 (ORS-5) and Sapphire – also contribute to the mission.

RSO collections are split between a tasking and scheduling approach in which various sensors are tasked to collect on RSOs of interest in a centrally managed process, but actual observations are scheduled by the sensors. Data is processed through the Space Defense Operations Center (SPADOC) and Correlation, Analysis and Verification of Ephemerides Network (CAVENet) networks. These systems represent the first challenge to maintaining SDA. Developed as far back as the 1980s, these systems and processes have long approached end of life, described by the Air Force Space Command (AFSPC) Commander in 2017 as “limping across the finish line until we can get [a new system] up” (Raymond, 2017). Processing capabilities are estimated to be 2–3 orders of magnitude below a mid-2000s Web server (Weeden, 2012). The most recent upgrade was canceled in 2019 as replacing the system without impacting critical operations proved difficult, as well as determining how to incorporate a major influx of new data (Clark, 2019). The USG is challenged from a technological perspective to maintain its own capabilities.

From the amalgamated data, RSO state information is re-calculated on demand for high-interest RSOs every eight hours for Near-Earth RSOs and every 12 h for Deep Space RSOs (Lal et al., 2018). Positional information is distributed in the form of Two Line Element sets (TLEs), without covariance information, on the public Space-Track database by the 18th Space Control Squadron (18 SPCS). However, this information is insufficient for many modern users.

The final two challenges impacting future SDA needs are the expansion of space utilization and the need to address new threats. Analysts project a growth in space services, with deployments of large constellations of hundreds of satellites already underway. This has in part been spurred by more frequent, affordable launch opportunities with technological advances in satellite capabilities. Increased adversarial threats are also prompting additional SDA needs.

Recognizing these problems, Space Policy Directive 3 (SPD-3) was released by the President in 2018 (Trump, 2018). This call to action asserts that “as the number of space objects increases…[the current] limited traffic management activity and architecture will become inadequate” and directs executive departments to pursue: improvements in observational data, algorithms and models; developing new hardware and software to support data processing and observations; mitigating the effect of orbital debris; improving SDA data interoperability; and enabling greater data sharing. Agencies were also directed to improve SDA coverage and accuracy by seeking to minimize deficiencies in SDA capability, “particularly coverage in regions with limited sensor availability and sensitivity in detection of small debris” through data sharing, data purchase, or the provision of new sensors; developing better tracking capabilities; and developing the standards and protocols for creation of an open architecture data repository.

Air Force Space Command Instruction 10–610 (AFSPCI 10–610), Space Situational Awareness Metric Data Integration Guidelines for Nontraditional Sensors, provides a defense-focused response to SPD-3 (AFSPC, 2019). The guidance outlines how to explore using nontraditional capabilities, stating that the “intent is to improve SDA through the utilization of a wide variety of sensor data and ephemeris data, of varying fidelity and accuracy, from an array of USG, non-DoD, commercial, civil and foreign data providers.” It also emphasizes that “the quality and compatibility criteria for new data sources should be set as broadly as possible.” Data challenges have been partially addressed by the creation of the Nontraditional Data Pre-Processor (NDPP) which allows ingesting data from new sources. The crux of this research lies on formulating an objective study to analyze these alternatives.

Extant nontraditional optical sensors

There are a plethora of telescopes with the ability of contribute to the USG’s SDA needs. An example of a civil sensor whose contributions may be easy incorporated is the National Aeronautics and Space Administration’s (NASA’s) 1.3 m Meter-Class Autonomous Telescope (MCAT) sensor on Ascension Island. Used to statistically characterize orbital debris, it has already been proposed for use as a contributing sensor to the SSN (Lederer et al., 2013). Additionally, Lal et al. found a “recent development has been the repurposing of existing sensors previously used for astronomy and other scientific research “for SDA purposes (Lal et al., 2018). More broadly, SDA data gathered incidentally by sensors performing related missions such as astronomy may be useful. Bellows demonstrated that RSO positional updates can be obtained using data from RSO streaks gathered serendipitously by astronomical telescopes, citing the Panoramic Survey Telescope and Rapid Response System (Pan-STARRS) and the Large Synoptic Survey Telescope (LSST) as capable of providing advantages to DoD (Bellows, 2015).

Multiple worldwide commercial networks rivaling DoD’s coverage at the expense of lower fidelity now exist. These capabilities came about in part by “perceptions related to lack of transparency with DoD data” and improvements in optical sensing technology with a reduction in parts cost (Lal et al., 2018). Major optical Deep Space cataloging companies include Analytical Graphics Incorporated (AGI), ExoAnalytic Solutions and the Numerica Corporation. As an example of commercial capabilities, ExoAnalytic offers an extensive list of services which includes simply purchasing its data or pursuing a turn-key package of one 14 in telescope with 1° NFOV capable of observing RSOs down to 18.5 M_v for $500K (ExoAnalytic Solutions, 2020).

Related studies

No study has considered improving the USG’s SDA cataloging in the manner proposed herein; however, related studies have guided this approach. Moomey successfully translated a Commander’s Intent to improve SDA cataloging into the simulation of a worldwide network of small rate-tracking commercial telescopes through the systems engineering process, demonstrating that adding 60 telescopes at each of five worldwide sites could allow up to 2 h of extra time per night on each GEODSS telescope (Moomey, 2015). Raley et al. highlighted the Defense Advanced Research Project Agency’s (DARPA’s) OrbitOutlook effort, demonstrating that data from Space-Track and multiple civil, commercial, academic and hobbyist providers could be combined to improve overall SDA goals while overcoming data surety concerns (Raley et al., 2016). Richard et al. developed a framework to design survivable radar satellites via Multi-Attribute Tradespace Exploration (MATE) using decision-maker input to identify attributes and weightings, propose design variables and realistic architectures and use modeling and simulation (M&S) to find Pareto-efficient architectures based on cost and utility (Richard et al., 2009). Limitations of Moomey and Raley et al. include failing to optimize the AN tradespace while Richard et al. omitted a large-scale M&S because of computational limitations.

Several studies exploring the development of a new, alternative network to the SSN are pertinent. Ackermann et al.’s, 2015 work coupled modeling and simulation with expert knowledge of telescope siting constraints to design a network to outperform the SSN’s Deep Space tracking capabilities (Ackermann et al., 2015). Colombi et al. sought to create the optimal network of SDA cataloging sensors from a large tradespace of ground and space-based telescopes by evaluating multiple architectures using MOO powered by a physics-based M&S of a nightly collection scenario (Colombi et al., 2018). Work by Bateman et al. (2018) and Felten et al. (2018) extended Colombi et al.’s work with an improved methodology, M&S and additional sensors. Although these studies were comprehensive and insightful, they neglected the reality that a clean-slate approach to USG tracking is unrealistic in the current fiscal environment.

The last area of study concerns inferring likely AN requirements and performance measures. USG SDA and SSN requirement documents are felt to best inform this consideration; however, no open-source documents on these subjects are found in literature so proxies are assumed. Moomey’s end-to-end system engineering approach with mission requirements and measures best illuminate the end-to-end requirements process and desired outcomes (Moomey, 2015). Daw and Hejduk’s, 1999 study on SSN operational capabilities identified three target areas with associated performance measures which lend themselves to this study: suitability, including accessibility and connectivity; effectiveness, including coverage, capacity, responsively and detectability; and performance, including timeliness and quality (Daw and Hejduk, 1999). Finally, AFSPCI 10–610 outlines desirable sensor criteria in Military Utility Assessment (MUA) criteria which are considered. These include accuracy, capacity, sensitivity, field of regard/orbital coverage, search rate, tasking responsiveness, unique capabilities, availability, reliability and cost (AFSPC, 2019).

Methodology

Figure 2 illustrates the methodology. The SPO purchase decision discussed in the Introduction section is addressed using Crawley et al.’s approach, which requires modeling and simulating architectural permutations, assessing them based on performance measures and using Pareto optimality to evaluate alternatives. Multiple days are assessed using methods similar to those of Epoch-Era analysis.

Develop representative sites and sensors

The nine 1 m GEODSS telescopes are chosen to form the baseline USG architecture; space-based capabilities are neglected in this study. Three separate hypothetical commercial companies with several 0.3–0.8 m telescopes are developed based on open-source literature review. The GEODSS and commercial sensors are assumed to operate in one centrally tasked, centrally scheduled manner performing the core USG SDA mission because of exclusive purchase rights. A sensor similar to NASA’s 1.3 m MCAT on Ascension Island is chosen to be a civil contributor, providing information mostly on debris. A hypothetical large-aperture Wide Field of View (WFOV) telescope, the Brazil-Internacional Gigante Global Observatorio (BIGGO), is developed as a scientific contributor which serendipitously collects on RSOs while performing an astronomy mission at a fixed sky position. All telescopes with the exception of BIGGO are assumed NFOV and use a task-based search method.

Hypothetical locations, sensor specifications and business cases are developed for the commercial networks based on review of open-source material and artistic license. Real-world telescope and camera specifications are used in the simulation. Table 1 summarizes the business cases.

Translate needs into measures

Moomey’s systems engineering approach is applied to identify possible AN requirements, from which many performance measures are derived. The measures are compared to those identified in literature and three are deemed common and practical for this study. Their algebraic formulations are covered in a subsequent subsection:

• Average capacity: capacity is defined as the number of times a particular RSO is observed by sensors in the architecture; the average value amongst all RSOs is computed.

• Average maximum time unobserved: time unobserved is defined as the maximum time an RSO goes unobserved during the scenario; the average value amongst all RSOs is computed.

• Coverage: coverage represents the ability of sensors to track regions of the GEO belt with managed redundancy. The ±15° latitude region defining the GEO belt is broken into 1° × 1° bins. Each bin is assigned a value based on the number of sensors capable of viewing it and a diminishing returns formula is used to discourage an architecture with all sensors in one location. The coverage measure is computed by summing the values in all bins.

The approach to calculating these measures has been refined over previous iterations of this work. High coverage was found to be easily achievable when cast as the ratio of observed RSOs to those simulated. A weighted geometric value is proposed to better capture viewability of RSOs while encouraging geographically dispersed, redundant capabilities. Average Maximum Time Unobserved is proposed to better report the temporal frequency of observations. Previously, average and maximum revisit times were used, unexpectedly causing large variations when adding new observations further in time from a core group and unfairly penalizing any RSO receiving an extra observation. The current approach minimizes this occurrence.

Model and simulation: supporting tasks

Figure 3 illustrates the M&S framework, which requires various supporting tasks prior to running the actual assessment. Colombi et al.’s basic M&S methodology is adapted and modified. Unless otherwise stated, all tasks are completed using Python with its Pandas database and NumPy numerical libraries.

First, 954 GEO RSOs and their TLEs are pulled from Space-Track on 7 Mar 2020. RSO names are fused with owner, status as active or debris, mission and standard Mv data from Celestrak, Gunter’s space database and Calsky. The RSOs are binned into five categories, with Category 1 being highest importance, based on analyst judgment and in proportion to percentages suggested by 18 SPCS in similar work (Dararutana, 2019).

Site latitudes and longitudes are stored in a separate file. LEEDR is used to estimate atmospheric transmittance and extinction values at all sites during summer and winter nights. A choice is made to handle clouds by merely blacking out a sensor for an entire night if a randomly generated number exceeds a threshold. This threshold, the Probability of Cloud-Free Night (PCFN), is developed using data from the 14th Weather Squadron (14 WS). Up to 30 years of sky coverage data from hourly meteorological reports is analyzed to estimate the average nightly cloud coverage at each site for every calendar month. A cloud-free night is assumed as a night with average sky coverage less than or equal to five oktas on a zero to eight scale. The proportion of cloud-free nights at each site for a given calendar month garners the appropriate PCFN.

Because weather is stochastic, several Monte Carlo simulations with different cloud conditions are designed. Each Monte Carlo is represented by a string of binary bits, where each bit represents a site, with one indicating cloud-free conditions and zero too cloudy to collect. The determination for a one or zero is based on the comparison of random-number generator (RNG) values to PCFN thresholds. A total of 200 weather simulations are deemed sufficient after a small set of architectures is tested using 10,000 Monte Carlos. Oberle’s percent error-based approach using the Central Limit Theorem states that the minimum number of simulations may be found by taking a sample with a large number of Monte Carlos, assuming the sample standard deviation and averages are close to the those of the population and calculating:

(1) m=(zα/2100sεYavg)2

for each measure, where z_α/2 is the z statistic, ε is the percent error, s is the sample standard deviation and Y_avg is the sample average (Oberle, 2015). For a 95% confidence interval (z = 1.96) and a two percent error (ε = 2), analysis of the set show that at least 90 simulations are necessary. A second calculation to illustrate the percent change of the measures’ averages as the number of Monte Carlos is increased shows the point at which the percent change is consistently less than 0.5% occurs with 200 simulations. A total of 200 simulations are felt to be appropriate.

Two STK scenarios are created containing all sites and RSOs; one for Vernal Equinox and one for Summer Solstice. These dates are chosen for evaluation because of their poor collection conditions; GEO RSOs experience a 1-h eclipse prohibiting detection on the former, while the shortest nights in the northern hemisphere occur on the latter (Colombi et al., 2018). On each date, RSOs are propagated forward in time over a 24 h period using the Standard General Perturbations 4 (SGP4) algorithm, which neglects covariance information and assumes a perfectly known RSO position. Sites are constrained to operate only in umbra and track RSOs at an elevation ≥20°, a solar exclusion angle of 40°, and a lunar exclusion angle of 10°. Note that sensor parameters are modeled later in STK.

An assumption is made that all sensor-RSO collections require 30 s to settle, take observations and slew to the next target. This allows the 24 h day to be broken into 2,880 finite intervals, greatly simplifying the scheduling discussed later. STK’s astrodynamics routines are then used to output various reports in 30 s intervals, including a list of all possible site-RSO accesses, an array of 1° × 1° coverage areas for sites and various quantities required for lunar brightness modeling.

All data sources are then combined to determine possible scheduling opportunities. Site-RSO accesses are expanded to include all sensors at each site. All sites are assumed to operate with a clear moonless night background radiance of 21 M_v/arcsec² while lunar and RSO intensity are estimated and incorporated separately via the approach previously outlined. Only sensor-RSO scheduling opportunities with SNR ≥ 6 are kept.

Three separate schedulers are run: combined GEODSS/Commercial, MCAT and BIGGO. For all three, a simplification is levied that the scheduler be run only once up front and observations from particular sensors be taken from a master list when doing architectural evaluations. This sidesteps running the scheduler on every architectural evaluation, which is time-prohibitive.

The GEODSS/Commercial scheduler is custom-designed to quickly schedule a prioritized list of RSOs for geometrically and temporally dispersed collections. Based on data from Space-Track and Calsky as well as analyst judgment, RSOs are sorted in a prioritized list in the following manner: extremely dim (approximately 20 RSOs), very old TLEs (approximately 20 RSOs, only on first scheduling pass) and Category 1–5 observations. Within each grouping all RSOs are sorted by number of available accesses to encourage less-available RSOs to be scheduled first. The RSOs are scheduled via a loop until no sensor-RSO collections remain. Each RSO is scheduled for a maximum of 20 observations at a time so as to prevent higher-ranked RSOs from receiving all possible collections. The 20 observations are spread evenly amongst the available sensors based on geography, then randomly assigned timeslots; this serves as a surrogate for collecting along multiple parts of the orbit, thus reducing RSO covariance uncertainty. At the end of the loop, the RSOs are sorted again in the same manner. This process of scheduling GEODSS/commercial observations requires around one hour on an Intel i5 Sandy Bridge laptop with two cores/four threads operating at 2.5 GHz with 8 Gb of Random Access Memory (RAM).

MCAT is scheduled using a similar routine. Because of its various missions, it is set to track only at the first and last hour of the night. GEO debris is prioritized; when complete, active satellites are collected. BIGGO is not scheduled per say; it is assumed to point at a fixed azimuth and elevation for the entire scenario performing an astronomical mission. Any RSOs passing within ±2° of this position of sufficient brightness is assumed to be serendipitously collected. This results in several GEO RSOs being collected for several hours. In lieu of reporting all data on these RSOs, only the first and last opportunities for each are returned as contributions.

Model and simulation: assessment

The simulation is run for both days, each day using Deb’s Non-Sorted Genetic Algorithm II (NSGA-II) heuristic routine in the Python Inspyred module (Garrett, 2012). All architectures assessed on one day are then evaluated under conditions for the other to permit a fair comparison. Using Python’s Multiprocessing routine to parallelize the evaluation of architectures in each generation, the simulation completes in 24 h on an Intel Xeon Sandy Bridge Workstation with 16 cores and 32 threads running at 2 GHz with 56 Gb of RAM.

NSGA-II is used as follows. As the nine GEODSS sites, MCAT and BIGGO are always used in the scenario, the only variable sensors in the architecture are the set of 56 commercial telescopes. An architecture is thus represented as a 56-gene binary chromosome, where the choice to use the sensor is represented by one or zero, for a total of 10¹⁷ possible AN permutations. For each day, five separate NSGA-II trials of 50 generations and 100 architectures each are evaluated. Randomly selected architectures meeting the cost constraint are used in the first generation. Each architecture is evaluated for the 200 weather conditions, and results averaged together for final architecture performance measures before crossover, mutation and selection occurs. All architecture costs and performance measures are saved for post-processing.

Each architecture evaluation solves the following pseudo optimization problem:

(2) Maximize c={1m∑j=1m(∑b=1b(∑s=1s1s)),  d≤25       penalty,          d>25

(3) Maximize p¯={1m∑j=1m(1n∑i=1nri),  d≤25     penalty,      d>25

(4) Minimize l¯={1m∑j=1m(1n∑i=1nmax(Δt0…Δtf)),  d≤25          penalty,               d>25

where c is coverage, s is the sensor capable of observing the 1° × 1° bin, b represents all bins covering the GEO belt, m is the number of Monte Carlos, p̄ is average capacity, r_i is the observed RSO, n is the total number of RSOs observed by the architecture, l̄ is the average maximum time unobserved, and Δt represents all unobserved durations for each RSO. A large penalty is applied if the $25M cost constraint is exceeded, making the value unfavorable to the optimizer. The underlying values are computed by conducting operations on efficient arrays based on the list of scheduled observations.

Findings

All architectures and top 10% performers

In total, 50,000 architectures are evaluated in total, of which 68% are feasible because of the cost constraint. For each date, the feasible architectures’ measures are sorted using a modification to Blank and Deb’s fast non-dominated sort routine from PyMoo (Blank and Deb, 2020). Architectures are assigned a Pareto Front Ratio (PFR) based on their relative ranking amongst the fronts, where one indicates architectures on the best front. Figure 4 illustrates results on Vernal Equinox, where the average maximum time unobserved and average capacity are plotted with a color spectrum for coverage. The highest performers are those in the bottom right colored purple. Results for Summer Solstice are holistically similar.

Figure 5 plots individual architectures by PFRs on each day with a spectrum color-coding by cost, with higher cost approaching purple. A line representing architectures with equal PFRs on each day is also plotted for comparison. Architecture performance on both days is roughly proportional; however, deviations of up to 40% are observed. This illustrates the necessity of testing architectures on multiple dates. Cost is seen to increase as higher-performing architectures are encountered, which is intuitive as adding more sensors improves capability.

Top performers are those architectures with low PFRs on both dates; a determination is made that only the 187 architectures with PFRs ≤ 0.05 on both days be considered in the rest of this analysis. These architectures cost a minimum of $23.5m; 81% cost at least $24.5m. Each architecture consists of 33–35 commercial sensors. In total, 16 specific sensors of the 56 available (29%) are used in over 80% of the top performers, while two are used in less than 20%. It is difficult to make sweeping conclusions about sensor performance in the larger problem as the architectures are only a small sample from the large trade space. Factors contributing to the selection results include weather conditions, the ability to improve geographic diversity of the GEO belt, and the availability of GEO RSOs over sensors.

Figure 6 illustrates the performance of the top architectures. Around 49–51 average observations per RSO at a 595–630 average maximum time unobserved is achieved with global, redundant coverage. Average Capacity and Average Maximum Time Unobserved are better on Vernal Equinox than Summer Solstice, which may be expected because of Summer Solstice’s shorter nights which minimizes the number of collections. The differences in coverage are because of different cloud conditions during the different times of year.

Best performers based on additional considerations

Additional technical and managerial considerations may be applied to further reduce the decision space. This includes quantifying the realization of performance measures; pursuing minor cost savings and award fee equity; and considering sensor utilization per company, which is a proxy for business sustainability.

A simple indicator of better performance is found through the sum of PFRs measure, ΣPFRs, which sums each architecture’s PFR on both days; lower values are more favorable. The Percent Unrealized (PU) metric measures how far each architecture’s performance measure values are from the best possible amongst the top performers such that:

PU= |best−achieved|best x 100

Architectural comparisons are drawn by creating a sum of sum of PU (SSPU) metric for all three metrics on both dates; lower sums indicate better relative performance.

The cost savings under the $25m constraint is tabulated for consideration. Additionally, the equity in award fee is considered as it may help a decision-maker understand perceived fairness in competition. Finally, the sensor utilization equity metric is created and defined as the ratio of sensors from a company to those in the company’s proposal. This is felt to be a proxy for a company’s business case fulfillment. Taking the standard deviation of the three award fees σ_awd and the standard deviation of the three sensor utilization equity values σ_util, then looking for the lowest indicates those architectures with better value. Figure 7 illustrates the 19 architectures generated from these considerations which either fully capitalize on one of these metrics or attempts to balance them in some manner. Thus, a further reduction in the decision space is achievable with modest additional considerations.

Five sensors are common to all architectures. Table 2 provides specific information on the best architectures for decision-maker action. The five metrics are color-coded columnwise based on the performance of all 187 top architectures using a red–yellow–green spectrum, where greener cells indicate better value for that metric.

Comparison to baseline USG architecture

Table 3 compares the baseline as-modeled DoD architecture of nine telescopes to the top performers of the AN. A three-fold improvement in capacity and coverage and a 16%, or nearly two hour, reduction in maximum time unobserved can be achieved using the AN concept.

Conclusion

This study has addressed a potential solution and methodology to improve the USG’s SDA cataloging dilemma. A hypothetical scenario in which nontraditional sensor contributions are added to the SDA architecture was developed and resolved via the system architecting process. Performance measures were identified, extant capabilities were deduced and modeled, and physics-based simulations of multiple AN permutations were accomplished through the aid of optimization. The results show that top-performing architectures can provide a three-fold increase in coverage and capacity with a 16% reduction in maximum time unobserved by adding two contributing sensors and 33–35 geographically dispersed, fully taskable commercial sensors at a cost of around $25m.

To quote a renowned expert in the field, “all models are wrong, but some are useful” (Box, 1979). As Crawley et al.’s lower-fidelity architecting strategy was used in this effort, these results are perhaps best used to illustrate the potential benefits of this methodology in tradespace exploration. To that end, various improvements are anticipated in future work. First, the inclusion of the ground-based SST and various space-based DoD platforms, as well as a hypothetical fourth commercial company conducting space-based collections, will add to the reality of the problem. Additionally, improving the fidelity of the optical collection model will improve confidence in the intermediate calculations. Finally, treating RSO positions stochastically, instead of assuming perfect orbital knowledge as conducted herein, will better simulate the challenges of RSO collections during sensor scheduling and ultimately result in more realistic performance measure values.

Figures

Figure 1.

Optical collection chain (the red RSO is collected by the blue telescopes only if SNR constraints, determined by supporting parameters, are fulfilled)

Figure 2.

Methodology (the SPO decision is addressed by developing representative capabilities based on literature, performing system architecting, running the M&S and assessing results)

Figure 3.

M&S framework (architectures are assessed on two different days based on information generated by various supporting tasks)

Figure 4.

Architecture performance on vernal equinox; higher performers are on the bottom right

Figure 5.

Relative architecture performance on both days (higher-performing architectures appear in the bottom left; cost increases with increasing performance)

Figure 6.

Architecture performance variation on each date

Figure 7.

Sensor use per architecture

Table 1.

Hypothetical company capabilities and business cases

Company (#Sensors)	Business case
1 (12)	• Limit number of sites/sensors and use mostly 0.3 m telescopes ($1m) • Offset high costs w/expensive, exquisite capabilities in high-interest locations • Use 0.6 m in Teide, Canary Islands ($1.5m); 0.6 m in El Leoncito, Argentina ($1.5m); and 0.8 m in Perth, Australia ($2m) • A discount of $0.2m is awarded if either all three large telescopes or all telescopes in the network are picked
2 (21)	• Mix 0.3 m and 0.4 m with different FOVs • Charge more for 0.4 m ($0.75m) than the 0.3 m ($0.50m) • Charge $0.25m extra for any site in the southern hemisphere
3 (23)	• Charge a low cost for a standard sensor (0.40 m, $0.50m) • Charge more for its locations in Israel and India (0.6 m in India) • Charge additional $0.2m if multiple sensors are used at one site

Table 2.

Best architectures and values for the additional metrics

Arch	Savings ($M)	σawd	ΣPFRs	SSPU	σutil	Notes
2	0.4	3.660	0.082	22.045	26.314	Very good σawd
9	1.45	4.003	0.081	23.623	27.658	Very good savings
13	0.25	3.739	0.058	27.705	25.066	Best σutil
21	0.45	3.573	0.069	23.624	25.370	Best σawd, Very good σutil
27	0.55	5.161	0.013	13.231	36.133	Best ΣPFRs, Best SSPU
31	0.05	5.310	0.013	18.044	34.954	Best ΣPFRs
36	0.4	3.632	0.082	23.640	26.879	Very good σawd
40	1.25	4.570	0.069	23.399	31.543	Good savings
48	0.7	3.653	0.081	22.317	25.977	Very good σawd
50	1.55	4.914	0.076	23.609	33.885	Best savings
53	1.4	3.925	0.076	24.981	28.364	Very good savings
63	0.25	4.463	0.019	17.133	30.661	Very good ΣPFRs
78	0.2	4.174	0.013	14.681	28.995	Best ΣPFRs, Very good SSPU
111	0	3.649	0.043	17.091	26.478	Very good σawd, Good SSPU/σutil
123	0.25	3.854	0.037	18.424	26.989	Good σawd/ΣPFRs/SSPU/σutil
129	0.15	4.124	0.019	16.751	29.416	Very good ΣPFRs
159	0	3.649	0.056	19.799	25.370	Very good σawd/σutil
161	0.45	4.385	0.019	16.529	30.566	Very good ΣPFRs
168	0.05	4.139	0.013	25.979	28.880	Best ΣPFRs

Table 3.

Comparison of DoD architecture to AN on both dates

	Avg capacity (#obs/RSO)	Avg max time unobserved (min)	Coverage (sr)
Vernal Equinox
DoD	16.8	707.0	11,213.1
AN	51.5	592.1	31,677.1
Change	206.5% (3.1x)	−16.3% (−115 min)	182.5% (2.8x)
Summer Solstice
DoD	16.1	723.9	11,317.1
AN	49.6	624.5	32,174.3
Change	208.1% (3.1x)	−13.7% (−99 min)	184.3% (2.8x)