DONUT : optimisation de missions aéronautiques et spatiales sous incertitudes météorologiques
Florent TEICHTEIL-KOENIGSBUCH - Airbus AI Research
DONUT paves the way for future aerospace missions
Mastering uncertainties (e.g. weather, demand, traffic, hardware) in mission planning (e.g. flight, fleet) using AI decision-making algorithms for:
● More efficient resource management (2020+)
○ for aircraft: improved fuel policy, higher PAX or range, improved network efficiency
○ for satellites: reduced image delivery delays, reduced image rejection rate
● Towards autonomous operations (2030++)
○
for aircraft: single pilot operations, autonomous aircraft○ for satellites: autonomous decision-aiding system for operation centers
Methods: Probabilistic Decision-Making (MDP, RL, robust scheduling, …) with learned
probabilistic weather and traffic forecasts
Integr ation
track BU/CR
T
DONUT in a nutshell
Resear ch tr
ack
CRT/ANU /UC3M
Reduce fuel reserves to 3% and
fuel burn by 2%
Take the next decision based on the context and on the
decision-aiding system
Sense the environment, the mission status and
the system health
Execute the decision sent by the control
center
Learn the operators decisions and uncertain
events dynamics Update the decision tree
branched on possible upcoming scenarios given
the context
plan/schedule under uncertainty
Reduce delays on image delivery to
clients by 10%
Send the next best decision to the aerial platform
Probabilistic Flight Planning
Pre-Flight
Ensemble Weather Forecast
Traffic Data
Probabilistic Model of Events
Impacting the Flight
In-Flight
average fuel
average time
Live
observations
2 modes:
● Robust Flight Plan
○ Static trajectory
○ Guaranteed bounded risk
● Dynamic Flight Plan
○ Event-based trajectory
○ Proactively modifies the trajectory to mitigate upcoming risks
Continuous update (dynamic mode)
controlled risk of flying through
"avoid" area
Probabilistic Network Management
average fuel
average time
average fuel
average time
average fuel
average time
average fuel
average time
In-coming flights Out-going flights
reallocate
● absorb delays
● minimize risk of missed connections
shifted departure time constrained arrival time update trajectory in flight
controlled arrival time
Ensemble Weather Forecast
Traffic Data historical observations
+
week predictions
aircraft allocation & flight schedule robust
weekly optimisation
Real-time flights & fleet re-optimisation
Sequential Decision Making under Uncertainty
Expected output: conditional plans or schedules accounting for uncertain events using Markov Decision Processes
ENVIRONMENT
AIR VEHICLE STATE
weather, ATC, demand, system’s health, ...
position, heading, target completion status, ...
REWARD
-δfuel, -δdelay, … δ targets, δpayload, …
ACTION
next waypoint, speed/altitude change, diversion decision, ATC communication, next imaged mesh, image upload, ...
state action state action state action
Markov Decision Process model
26/06/2018 7
Initial State :
- Departure Airport, - Initial Weight - Initial Time - Current weather - …
Decision : Choose of next waypoint to fly over, target speed, target altitude.
Many possible outcomes:
1) Impossible decision ? (extreme weather, ATC constraint)
2) Accepted transition : Possible weather scenarios (temperature and wind influence the next state !)
Run flight model between two waypoints
Optimize : Aggregation of :
Fuel Cost, Time Cost (Crew + Delay), Maintenance Cost (if modelled)
2018 aircraft/airline use case results
● Demonstrator I
○ 2D probabilistic ESAD optimization without ensemble forecasts (but historical observations)
○ 200+ city pairs
○ ESAD reduction from 0.5% to 1.5% depending on flight range
○ fleet sizing optimisation algorithms solving LATAM
○ weather forecast (deep) learning proof of concept
● Demonstrator II
○ 3D probabilistic fuel optimization with ensemble forecasts and fuel reserve constraint
○ CRT algorithm: fast but sub-optimal
○ ANU algorithm: optimal with guarantees but slow
○ Iberia long-range flights
○ advanced fleet sizing optimisation algorithms from ANU solving LATAM and JETBLUE
“God”’s (foresight) plan DONUT In-service flight planning technos
Robust approach (optimization in hindsight)
OPTIMISTIC MEAN/MEDIAN ROBUST Take off policy
Hindsight Optimization - Theory
Approximate MDP solving by sampling scenarios and averaging deterministic solutions (one per scenario) Used either to break MDP complexity or when probabilistic transition functions are unknown but
scenario-based transitions (our case)
Idea: approximate Q-value Q(s, a) by generating one successor state per scenario from which a deterministic planner is called then averaging over all deterministic plan’s values
Issue of standard approach: too large action space in our case
Solution: generate helpful actions as best first actions to execute from s in each scenario according to the deterministic planner
10
Complete optimal approach (C-SSP): problem description
•
•
•
•
• 1.
2.
3.
11
Assumption: we have access to a weather forecast model
Column Generation
Operations Research technique guaranteed to find the optimal solution.
Take-off Landing
Expensive
Very cheap
•
•
•
•
•
•
C-SSP Results – Deviation Airports and Late arrival
12
Probabilistic vs Deterministic Flight Planning
Space use case architecture
Probabilistic Planner
Set of imaging requests
New request (large area to acquire)
Satellite Operator
Updated schedule
+
Execute 1st tasks
Update set of pending requests knowledge reasoning
weather planned
advice
operator’s decisions observe
& learn
costs
Data to be provided by ADS
User imaging requests PROPHeT simulation CRT work
Cloud processing
Markov Decision Process model
13/05/2018 © Airbus - Confidential 15
Set of image meshes completion status, image request priorities, satellite position, time
Decision: update priorities Many possible outcomes:
sets of image meshes completion status, image request priorities, satellite position, time
1 orbit = 1 PROPhET run
All requests done, compute delays cost (Tend - Tcom)
2018 space use case results
● Demonstrator I
○ (Asymptotically) optimal deterministic algorithms
○ Small artificial backlog (4 requests)
○ Preliminary results with probabilistic optimisation-in-hindsight algorithm
○ Design & testing of various optimality upper bounds
○ Up to 6x gain compared with simple operator baseline but not robust (high variance)
○ handover of SANTO algorithm to ADS (with patent)
● Demonstrator II
○ Probabilistic meta-heuristic algorithms with divide-and-conquer heuristics
○ Realistic artificial backlog (5000+ requests)
○ ANU cloud coverage data analysis highlighting need for ensemble forecasts instead of observations
○ 25% reduction in image delivery delays on average compared with simple operator baseline
Image requests
Uncertain cloud coverage
“God”’s (foresight) plan
DONUT
baseline
Airbus / Météo France collaboration - 2019 internship
Météo France point of contact: Laure RAYNAUD
AIRLAPS: AIrbus RL, Automated P&S toolbox
Towards an AIRBUS-initiated framework for planning/scheduling and reinforcement learning library
Focus on "problem-solving" point-of-view (no need for user to be an algorithmic expert)
Embeds existing and future problems and solvers inside a common API
Not yet another sequential decision-making library!
Non-confidential problems and solvers to be opensourced to benefit from academic worldwide momentum
Plug & Play philosophy: e.g. Airbus problems plugged to the solvers library for
internal use but not opensourced
18© Airbus - Internal
AIRLAPS framework: Domain definition
19
Partially
Obser vable
Fully Obser
vable
Full Hist
yor
Finite Hist
yor
Marko vian
Memor y-
less
Goal
Initializable Uncer tain
Initializ ed Deter- ministic Initializ ed
Rewar
d
Positiv
e Cost
Events
Actions Unr
estricted Actions Environment
Simulation
Uncertain Transitions Deter- ministic
Trans- itions
Dynamics
Events
Value
(Initialization) (Goals)
Memory Observability
© Airbus - Internal
AIRLAPS framework: Solver definition
20 Restor
able
Policy
Uncertain Policy
Deter-ministic Policy
Utilities Q values
Solution
Anytime
Realtime
Temporality
(Assessability) (Policy)
(Restorability)
© Airbus - Internal
