Asteroid is a lightweight Mac app for AR development that makes it easy to connect computer graphics and machine learning. It supports Apple's CoreML models, and can be used with Python scripting. For example, you can use the output of a machine learning model to control the trajectory of a 3D object, or take the output of a camera and feed it into your computer vision script.
  • Scene Objects

    We can add a number of visual objects to the 3D scene:

    • GameObject - a blank object to which you can attach scripts, etc
    • Light - illumination for the 3D scene
    • Shader - a GLSL shader for rendering an object
    • Text Sprite - a 3D representation of text
    • Camera - a perspective from which the entire scene is rendered
    • RenderObjects - a 3D object froman imported mesh file
  • Inputs and Outputs

    Asteroid provides a number of I/O objects for accessing sensor data or information from external programs. The data from these objects can be used to control the behavior of Scene Objects, or to use as input to a machine learning script.

    • CameraInput - object for accessing the video frames from your Mac's iSight camera. Also used in conjunction with the ARTracker object
    • Tracker - attach this to a camera input object to track faces or planes.
    • MicrophoneInput - object for pitch/amplitude detection; accessing raw audio-buffer and FFT data from your Mac's microphone
    • OSC Input (beta) - Using the Asteroid Module, you can set up your script to receive input from external applications using the Open Sound Control protocol
    • OSC Output (beta) - scriptable way to send OSC data from Asteroid to an external application

    To learn about using machine learning with Asteroid, check out our document on some of the available Python-based machine learning libraries

  • Scene Graph

    Click the "Scene/Graph Mode" toggle button in the toolbar at the top to switch to graph mode. This manages relationships between objects in the Asteroid scene. A relationship between two nodes means that the second node is synced to the value of the first node, every frame.
    For example, if the Mac's camera output is connected to the input of an ML model, every frame the model will be fed a new still image.

  • Imported Assets

    Once we load the working folder, icons representing the files in the folder appear. Asteroid will try to create an object of the correct type (e.g. a mesh file will be added to the scene as a RenderObject).

  • Augmented Reality

    Asteroid scenes can be exported into Xcode projects, which can be run as ARKit-ready apps on your iOS devices. In the top toolbar, select "Asteroid-> ARKitProjectGeneration" to copy all the assets and Swift scripts from your Asteroid scene into a generated Xcode project.

  • Scripting

    The other important part of Asteroid simulations are scripts. There are two major types of scripts; PreProcess, PostProcess or a Controller.

    • PreProcess scripts are run before the simulation begins
    • Controller scripts run every frame during the simulation
    • PostProcess scripts act on the rendered simulation
    • Machine learning objects handle CoreML models

    These scripts are written in Python. For more detailed information, check out the scripting guide

© 2018 by Asteroid. All rights reserved.