Reality Studio is a set of two applications and a Unity Editor tool developed solely by me. Its goal is to create an environment where interior design layouts can be quickly and easily created, changed, and viewed in a high fidelity AR experience by someone who is inexperienced with traditional 3d software. The UI/UX was designed and drawn by Jessica Lin.
The first application is called Content Creator, it is the 3d editor that allows the user to edit and manipulate an interior space. From there, a saved project can be uploaded to the second part of the pipeline. This is the Unity tool that uses AWS, Unity’s Lightbaking, and Unity Addressables to bundle the scene for mobile use. Once the project has been processed, it can be viewed within an AR environment on a phone or tablet.
The linked Git repo will only contain individual systems out of context rather than the complete project.
Content Creator Overview
The ‘Content Creator’ app is a PC Unity application that functions like a 3d editor (similar to Unity itself). It contains a multitude of tools used to aid the user in creating an interior space. Projects can be saved and loaded, and once finished, uploaded to AWS for the processor to take over. Content Creator includes the following systems, some of which were from asset packages, but most made by me:
- Transform Gizmos
- Object and Surface Selection and Multi-Selection
- Change History (Undo/Redo)
- Material and Preset Management
- Mesh and Submesh Manipulation
- Custom Project File Saving/Loading
- Square Footage Calculator
- Menu, Tutorial, Hierarchy, and Inspector UI
Selection
Content Creator has object selection, surface selection, and supports multi-selection. The Selection Manager is the go-to source for the currently selected object, which can be accessed from anywhere. Selection is done through either a raycast, or through the object hierarchy. If selected via a click, it will first select the root parent object, and then allow for click selection of the child objects. Selected objects and surfaces are highlighted for easy viewing.
While object selection is simple enough, it also supports a surface selection mode, where a submesh is determined and selected. The submeshes do not have their own transforms, but they can be split off into their object with a command, and the submesh’s material can be changed through the inspector.
Both object and surface selection supports multi-selecting (and multi-editing). The Selection Manager internally stores all multi-selected objects and/or surfaces and displays all their components in the inspector. When changes are made, it uses reflection to update the data of the selected objects in whatever manner is appropriate.
Mesh Manipulation
In order to standardize imported meshes, I created a system I call the ‘Mesh Slicer‘, as well a ‘Mesh Zeroer‘. Since meshes can be wildly varied, with their center’s different and some parts conjoined where others may not be, these two scripts ensure that they all match, at least functionally.
The Mesh Zeroer simply moves the vertices of a mesh to the desired position with a new origin. This allows things to have their center point always be at the bottom, center, etc.
The Mesh Slicer is a complex tool that processes each imported mesh, iterates over them, and slices them into submeshes. This is a very complex process, but ultimately it decides where to slice based on the angle between two tris and the closest distance between the submeshes chosen by the first step. The mesh is regenerated, the UVs are remapped, and materials are multiplied. As this mesh slicing process can take some time, it was sped up using multithreading, and the result is saved so long as the application is still open, should the user need to import the same mesh again.
The image below shows a mesh with randomized colors after it has been sliced. Note how the cabinet handles and table tops are one solid color!
Undo/Redo
Undo and redo is a must have in an application like this. However, I went a step further and created a generic, reusable Change History system for any Unity-based project! The system supports not only changes made to classes and components, but also custom changes declared with delegates. When exactly to record changes is left up to the developer, but extensions off of every object type class make it very simple.
The system is very powerful and works for everything I have added to Content Creator so far. Its base use case uses reflection to update values stored in classes.
Material Management
Material management and assignment is one of the core aspects of Content Creator. Rather than making and editing individual materials, there exists a Preset Library of material Presets. This preset library can be added to and edited, and it also supports any type of object, such as lights. Materials from the library can be drag and dropped into material variables the inspector.
Saving/Loading
Saving and loading went through two iterations. The first saved relevant data to a json file and re-loaded any models and textures that were also stored in the same folder structure. This method was a bit messy, as all the parts and pieces of the scene were scattered between multiple files, and meshes had to be re-processed. The second iteration sought to save and load every piece of data contained with in the scene dynamically all into one single json file. This process turned out to be extremely complex, and I was essentially writing my own data serializer with conditions for every little quirk of Unity and C#. Unfortunately, the second iteration of the save system was not fully finished due to the project being put on hold. It was able to, however, successfully save and load transforms, material presets, renderer settings, entire meshes, and everything else. The only remaining issue was a multitude of bugs.
UI
There is quite a lot of UI in Content Creator. It was designed to feel close to Unity itself, although with different commands and a simpler interface. All UI/UX and sprites were designed and created by Jessica Lin, while I did the in-engine implementation and functionality. The Hierarchy and Inspector was an asset from the Unity asset store, however it ended up being used as a framework for my own code, so very few of the components of it are default from the package.
The functionality of the menus and buttons were all done by me, using some specific menu controllers, and more generic UI scripts, such as this Radio Button one. Most button click functionality links directly into the Project Manager, which handles many aspects of Content Creator.
The hierarchy and inspector UI is linked to selection, and handles the display of all relevant information about the selection. A lot goes into the various Inspector Drawers to control the data visible to the user.
Lastly, an application of this caliber would be quite confusing without proper guidance, so I also created a drag-and-drop Tutorial System to guide the user around how to use the application on their first use.
Processor Tool Overview
The second step in the chain is the Unity Editor tool, the Processor. This is intended to be an editor script that runs on an instance of Unity on a cloud server. It’s objective is to prevent the need for a user to bake lighting and bundle scenes for mobile use. It ticks once a second, checking for new files on an AWS server. If one is found, it downloads it, unzips it, and using the same save/load system, recreates the scene inside the editor. It will then unwrap all the UV’s, set all of the light and render settings to their appropriate settings, and then bakes the lightmaps. It then creates an Addressables asset bundle and uploads it to a folder on AWS, where it can be fetched from the AR app.
AR App Overview
Once a project has been lightbaked and bundled by the processor, the user can input the project name and download it via the mobile app for AR viewing. Addressables loads the scene, it is placed into the AR plane, and the user can walk around the environment! The AR app is a modified version of another client project used for viewing homes in augmented reality, so unfortunately it’s scripts will be omitted.