ReFlex

Software Development SDK for Elastic Displays as open source mono repo

Complete Documentation as github pages available at https://visual-engineers.org/reflex/

• .NET Core library as framework for different depth sensors, calibration, depth image filtering and reconstruction of interactions
• APS.NET Core / Angular server application as frontend for library
• can be packaged as electron app for desktop
• Example client applications for Plain HTML, Angular, React, Vue.js, Plugins for Unity, Unreal Engine 5
• Emulator as Development Tool
• Example Applications

Table of contents

1. Build status
2. Repository structure
3. External Dependencies
4. use shared code
5. NPM commands
6. Python gRPC Processing service
7. Known issues / Troubleshooting
8. Documentation (github pages)

Build status

ReFlex Library (.NET)
Shared Types (TypeScript)
Shared Components (Angular)
Tools/TrackingServer (Angular)
Tools/Emulator (Angular)

Repository structure

⬆ back to top

NPM workspaces

The repository uses NPM workspaces for better management of dependencies

Therefore, the repository should be initialized in the root folder by running the command npm install and not in the sub directories (although there are package.json files, but these files are just handling the local dependencies)

⬆ back to top

Repository initialization

• run npm run build:shared-types to build shared types library
• run npm install in root directory
• if building with electron: Install Electron-Builder globally by running: npm install electron -g

⬆ back to top

Development

• Adding a new workspace: npm init -w ./path/to/workspace/directory
• Adding packages to workspace: npm install -w ./path/to/workspace/directory package --save (in workspace root directory)

⬆ back to top

External Dependencies

• The following dlls need to be placed in the external directory for use with the associated depth cameras
• Intel RealSense R2/D435/L515 Intel.Realsense.dll, libpxcclr.cs.dll, libpxccpp2c.dll, realsense2.dll from Intel RealSense SDK (Files are included as nuget package)
• Microsoft Kinect Microsoft.Kinect.dll, Microsoft.Kinect.xml from Microsoft Kinect for Windows SDK 2.0
• for compatibility reasons, no sensors are included in the build process by default. To include specific sensors use the appropriate flags described in the Server documentation

⬆ back to top

use shared code

• for using reflex-shared-types in another project, just install it as workspace dependency in the current project with

    npm install ./packages/reflex-shared-types -w ./tools/ReFlex.TrackingServer/ClientApp --save
  

• types are available by importing @reflex/shared-types

⬆ back to top

NPM commands

Known issues

• Electron seems not to be perfectly suitable to be used in monorepos, as building the app in the package process removes all dev dependencies, including the electron-builder package if installed locally
  Therefore, electron-builder needs to be installed globally before executing a build:emulator:electron-xxx script
• Additionally, npm install is executed after packaging to restore the dev dependencies
• if a command build:emulator:electron-xxx is executed from within the emulator project, npm install has to be executed manually afterwards
• test and build-complete commands start a headless Chrome instance and therefore require Chrome to be installed and path to Chrome binary to be set in PATH.

⬆ back to top

Python gRPC Processing service

• as an example for integrating external services, a python gRPC service for extracting interactions from the camera depth image is provided in services/python-backend
• service is consumed when setting Interaction Processor to Remote
• in the ReFlex.TrackingServer, the service is configured in Services/RemoteInteractionProcessingService.cs
• Documentation: Python gRPC service

⬆ back to top

Known issues / Troubleshooting

• Karma Test Explorer Plugin for vs code does not work well with the current npm workspace setup, as it does not identify the correct angular path. In order to use the plugin, the global angular installation is used as fallback.
• if an application behaves different when executing the packaged electron version (either installed using the setup or the executable in the win-unpacked directory)), this may be caused by outdated Electron Cache. In this case, open
  • %AppData% Folder (Windows)
  • ~/.config Folder (Linux)
  • ~/Library/Application Support/ (MacOS)
    and delete the app folder there

• to delete all temporary folders / cached packages / build artifacts, run the following command in the scripts folder:

   ./cleanup_packages.sh package-directories.txt
  

⬆ back to top

Documentation (github pages)

Documentation is hosted as github page at https://visual-engineers.org/reflex.

The documentation content for the github pages can be found in the docs folder.

repository readme files

readme files in the repository (including this file) are copied into the docs directory as part of the deployment CI step by executing the scripts\copy_docs.sh shell script.

The related readme files are:

Testing github pages locally

testing the github pages can be done by following the instruction at Github Docs: Testing your GitHub Pages site locally with Jekyll

⬆ back to top