Automate unstructured tasks with skills based visual programming and orchestration platform
A large share of manufacturing is still difficult to automate. Robots work well when products are consistent, positions are fixed and the sequence is known in advance. But once tasks become more variable, perception heavy or exception rich the engineering effort rises quickly. This is a typical challenge we explore in ROB4GREEN, which develops flexible robotic systems for processing products after their first life. The project addresses real cases in automotive electronics, tyre retreading and wind turbine blade recycling, where variability makes conventional automation difficult.
Flanders Make is developing the Skill Based Orchestration framework, or SBO, to help tackle this. SBO turns robot motion, perception, manipulation and other capabilities into reusable skills that can be visually combined into an application. Within ROB4GREEN we have already tested this approach for Valeo usecase for PCB desoldering, bringing robot control, RGB vision and thermal vision together in one workflow as shown below:
In the future the SBO framework is going to be used in Michelin pilot for tyre retreading and Acciona pilot for Wind Turbine Blade Recycling.
Beyond the main ROB4GREEN demonstrations, SBO will also be used in the open call pilots FlexiBAT, AGRI-SORT and PARTNER. FlexiBAT focuses on remanufacturing end of life EV battery modules into second life battery storage systems. AGRI-SORT develops an intelligent robotic cell for sorting recyclable material from mixed municipal waste. PARTNER focuses on human robot collaboration in material recovery facilities. The SBO will support the orchestration of adaptive robotic actions in real world environments.
What if robot applications could be composed from skills instead of programmed from scratch?
SBO is a visual programming and orchestration hardware agnostic framework which enables building goal driven robotic applications that adapt execution based on live data (vision, perception, sensors etthrough behavior trees. Engineers can combine perception, robot motion, manipulation and process logic using behavior trees. The underlying hardware is kept separate from the application logic. Robots, cameras, tools and AI models are connected through reusable plug ins. This makes it easier to change hardware or reuse part of an application without rebuilding everything.
This approach also fits the broader ROB4GREEN work on enhancing robot skills through learning and adaptation. The skills can also improve over time by monitoring how the system performs. Failures and edge cases would be used to make the skills more robust and reactive. These improved skills would then be used by the same orchestration framework and can be reused across different workflows.
From individual robot skills to complete adaptive workflows
At the core of SBO is a modular architecture that separates application logic from the underlying hardware. Robot controllers, cameras, tools and sensors connect through reusable drivers and plug-ins. The same application logic can therefore be adapted to different hardware without rebuilding the complete software stack.
The framework currently supports robot platforms including ABB, KUKA, Fanuc, UR, Stäubli and Doosan. Camera and vision interfaces include systems such as Daheng and Pickit. Additional plug-ins are used to connect grippers, tools, sensors and industrial IOs. Python plug-ins and APIs make it possible to add new perception and AI capabilities as the application evolves.
A key part of SBO is visual programming. Skills and primitives are composed into behavior trees, giving engineers a clear view of how the application behaves. Sequences, conditions and recovery paths can be modified visually. Existing parts of a workflow can also be reused in another application. ROS2 is used within the framework, while interfaces such as TCP/IP, MQTT, REST and OPC UA allow communication with other production systems.
On top of the foundational skills like manipulation, perception and tool changing, higher-level toolkits are being created for palletization, depalletization, adaptive disassembly, adaptive kitting etc. These toolkits can be improved and reused across projects. Physics-based simulation is also being tested with NVIDIA ISAAC SIM simulation to test robot motions and application logic. The longer-term idea is to build a growing library of reusable toolkits that can be composed, tested and adapted as production changes.
SBO can cut robot programming effort by over 50%, reduce commissioning and debugging time by at least 30% which has potential to enable twice as many automation variants.
Apoorva Oak
Apoorva, Senior project manager Robotics and AI, has 16 years of experience across semiconductors, AI and robotics. His work now focuses on flexible robotic platforms that combine perception, intelligence and reusable automation to tackle increasingly variable tasks. His broader vision is to move manufacturing from fixed automation toward an economics of adaptability, where automation can evolve as quickly as markets, engineering, products and processes do.