Extending edgertronic capabilities - Extending the web UI
| ← |
⌂ Home |
→ Extending or replacing the web UI |
This section has the information about extending or replacing the web UI of the edgertronic high speed camera;
Extending edgertronic capabilities - Extending the web UI Index
New in v2.5.2: An experimental feature allows developers to add CSS / HTML / JavaScript to the existing web user interface that is displayed in the browser window. The existing edgertronic webUI is rather unique and thus extending the webUI follows the same concepts used currently for providing configuration and status capabilities. Some day I hope to completely rewrite the webUI code, which will likely break compatibility with what is described below.
Basic approach
The CSS / HTML / JavaScript that ships with the camera to create the webUI defines 10 empty custom JavaScript functions. A python application extension (app_ext) can make a callback during initialization to add additional CSS / HTML / JavaScript to the webUI. The added code is appended near the end of the camera's webUI code so that your code can redefine those 10 empty functions. Some of these 10 functions will reference HTML code, including: an optional status box (which overlays the live view window), an optional custom tab in the settings modal, and a set of user configurable items that are displayed when the custom tab is selected. The HTML code will likely reference some CSS, including status box style information. There is also some very mundane code around showing, hiding, moving, and locking down the optional status box. All told, a real implementation will consist of around 400 lines of CSS / HTML / JavaScript code.
The custom webUI code interacts with the application extension you added to the camera by calling URLs your custom application exposes. A common set of custom URLs is to retrieve and save custom settings.
Working example
A working example ships with the camera which can be found at http://10.11.12.13/static/sdk/app_ext/app_ext_foo.html which is loaded by app_ext_foo.py that lives in the same directory. Copy both app_ext_foo.html and app_ext_foo.py to the top level directory on the SD card and power on the camera. I have a few debugging hints at the end of this wiki page. If functions in app_ext_foo.html are cut-and-paste from the camera's webUI code, then I am too lazy or too incompetent to document those functions. You will want to look at the existing webUI when you encounter problems to see how the existing webUI handles a similar capability. The camera's webUI code can be found by telnet-ing into the camera and looking in the /home/root/ss-web/templates directory. As a hint, read sanstreak.html first.
For the example I try to follow The Google naming conventions and I hope the code passes jslint.
Add webUI elements
The app_ext initialization method signature
__init__(app, cam, cinfo, register_url_callback, register_html_file)
has been extended to support a new callback - register_html_file, which takes the filename of the html file you put on the big SD card. When your html code is activitated, the camera will force all browsers displaying the camera's webUI to reload.
Custom startup invocation
The provided HTML code is wrapped in <div> elements with the style including either display:none</t> or the class including hide. Thus your HTML is not displayed until your JavaScript code causes those <div>s to be displayed.
The first of the ten custom JavaScript functions of interest is custom_window_onload_handler(). This function is called after the rest of the webUI is initialized. Here is a typically implementation for the custom webUI for the way cool new foo capability.
<script>
function custom_window_onload_handler() {
fooAddStatusBox();
fooAddSettingsTab();
fooGetConfiguration();
}
</script>
You can see that the startup code adds the foo status box and the foo tab and associated foo setting.
Custom settings modal
You can allow the user to configure custom settings by adding a new tab to the settings modal. The user will click on the webUI wrench icon to bring up the custom settings, then click on the new tab name you add, and voila, your custom settings will be displayed allowing the user to adjust the settings. When the settings modal is closed, the custom_setting_modal_closed_handler() function is invoked allowing you to save and activate the settings.
The following sample code is the JavaScript needed to add the custom foo settings tab and tab contents.
<script>
function fooAddSettingsTab() {
document.getElementById("customTabButton").innerHTML = document.getElementById("foo_tab").innerHTML;
document.getElementById("customTabContent").innerHTML = document.getElementById("foo_content").innerHTML;
document.getElementById("foo_content").remove();
$('[data-toggle="tooltip"]').tooltip(); // work around a strange bootstrap tooltip defect
}
</script>
Notice you are adding your tab HTML code into customTabButton and your settings HTML code into customTabContent division tags. This gets the settings placed into the DOM tree at the correct location. Search the camera's HTML code to get a feel for the overall settings modal layout and types of controls you can add to your custom settings tab.
Retrieving the current custom configuration
The configuration is retrieved by calling a user added URL, such as http://10.11.12.13/foo_get_configuration and for the example, it returns a dictionary. Allowing the user to have configurable items is optional; you can just add a status box if that meets your requirements.
Note that URL call is asynchronous such that the processing of the response is done after the response is received. This might mean the webUI has a blank status box when the user first browses to the camera, then the box is filled in quickly after the empty status box is displayed.
<script>
function fooGetConfiguration() {
url = "/foo_get_configuration";
$.ajax({
type: "GET",
url: url,
contentType: "application/json; charset=utf-8",
dataType: "json",
async: true,
success: function (fooConfiguration) {
console.log("foo configuration: ", fooConfiguration);
fooUpdateSettings(fooConfiguration);
fooUpdateStatusBox(fooConfiguration);
},
error: function(data) {
console.log("Get configuration failed");
}
});
}
</script>
The important code is the calls to use the retrieved values to fill in the custom settings and status UI elements.
Custom settings modal tab
The HTML to define the custom text on the settings tab modal is straight forward.
<div id="foo_tab" class="hide">
<a href="#customTabContent" data-toggle="tooltip" data-placement="top"
class="help-tooltip" title="Foo settings" data-toggle="none"
onClick="fooTabDisplay()">Foo</a>
</div>
Adding the tab to the existing settings modal is done by defining new HTML content for the customTabButton. This is done by the fooAddSettingsTab() function shown in the next section.
<script>
function fooAddSettingsTab() {
document.getElementById("customTabButton").innerHTML = document.getElementById("foo_tab").innerHTML;
document.getElementById("customTabContent").innerHTML = document.getElementById("foo_content").innerHTML;
document.getElementById("foo_content").remove();
$('[data-toggle="tooltip"]').tooltip(); // work around a strange bootstrap tooltip defect
}
</script>
fooAddSettingsTab() also adds the custom settings as described below.
Custom setting modal content
The layout for the settings is accomplished using a table. For buttons a callback is used to allow the color scheme to change. For numbers, the values are retrieved when the settings modal is closed.
<div class="hide">
<div id="foo_content">
</p>
<div class="show">
<table>
<tr>
<td><button title="Minimum foo" tabindex=-1
data-toggle="tooltip" tabindex=-1 data-placement="top"
style="font-size:16px"
class="btn btn-normal btn-info help-tooltip">Minimum foo</button></td>
<td style="text-align:left;"><input type="number" id="foo_min" minlength="1" maxlength="3" size="5" min="-25" max="130" onchange="fooSettingChanged()"></td>
</tr>
<tr>
<td><button title="Maximum foo" tabindex=-1
data-toggle="tooltip" tabindex=-1 data-placement="top"
style="font-size:16px"
class="btn btn-normal btn-info help-tooltip">Maximum foo</button></td>
<td style="text-align:left;"><input type="number" id="foo_max" minlength="1" maxlength="3" size="5" min="131" max="1300" onchange="fooSettingChanged()"></td>
</tr>
<tr>
<td><button data-toggle="tooltip" tabindex=-1 data-placement="top" id="foo_button" type="button"
title = "Foo auto mode for those that can't figure it out themselves"
class="btn btn-info pull-right normal-tooltip" style="width:100%">Auto foo</button></td>
<td><div class="btn-group pull-left" data-toggle="buttons-radio">
<button title="Good starting choice" id="foo_auto_mode1"
data-toggle="tooltip" tabindex=-1 data-placement="top"
style="font-size:16px" type="button" onclick="fooSetAutoMode(1, true)"
class="btn btn-info normal-tooltip">Do most of it</button>
<button title="You are brave" id="foo_auto_mode0"
data-toggle="tooltip" tabindex=-1 data-placement="top"
style="font-size:16px" type="button" onclick="fooSetAutoMode(0, true)"
class="btn btn-normal special-tooltip">Just don't do it</button>
<button title="Stop being lazy" id="foo_auto_mode2"
data-toggle="tooltip" tabindex=-1 data-placement="bottom"
style="font-size:16px" type="button" onclick="fooSetAutoMode(2, true)"
class="btn btn-normal normal-tooltip">Do it all</button>
</div></td>
</tr>
</table>
<br>
<div id="foo_version">Foo Version: </div>
</div>
</div>
</div>
Populating setting modal
When the user added URL that retrieves the current settings returns, the success function invokes fooUpdateSettings()
<script>
function fooUpdateSettings(config) {
document.getElementById("foo_min").value = config['min'];
document.getElementById("foo_max").value = config['max'];
document.getElementById("foo_version").innerHTML = "Version: " + config['version'];
fooSetAutoMode(config['auto'], false);
}
</script>
Custom status box
The camera's webUI in normal operation consists of the control box (which contains the green trigger icon) in the upper left, the capture status box in the lower left, and the save status box in the lower right. You can add your own status box and place it where ever you like. All the status boxes support auto-show and auto-hide based on mouse activity. You can also move the status boxes around and pin a status box so it doesn't move or get hidden. The status box you add will also follow this pattern.
The following sample code is the CSS + HTML + JavaScript needed to add the custom foo status box. The custom_window_onload_handler() functions calls
The full foo webUI extension example contains the additional code to show how to show, hide, handle moving a status box and pinning a status box. Hopefully you can include that code with a simple rename to get it working.
<style>
.foo-status-box {
position: relative;
width: 180px;
height:200px;
border: none;
border-radius: 10px;
background-color: rgba(91, 192, 222, 0.4);
-webkit-user-select: none;
-khtml-user-select: none;
-moz-user-select: none;
-o-user-select: none;
user-select: none;
}
</style>
<!-- Foo Status Box -->
<div title="click and hold mouse down to move the foo menu" class="foo-status-box help-tooltop" id="foo_status_box"
data-toggle="tooltip" data-placement="right"
style="display:none;position:absolute;bottom:120px;left:240px;">
<div style="top:5px; font-weight:normal" class="inlineRight">
<button data-toggle="tooltip" data-placement="left"
title="pin Foo status box" id="foo_status_box_unpinned"
type="button" class="btn btn-mini btn-info help-tooltip" style="display:inline"
onclick="pinFooStatusBox(true)">
<i class="fa fa-thumb-tack fa-rotate-90 fa-xs"></i>
</button>
<button data-toggle="tooltip" data-placement="left"
title="Foo status box is pinned" id="foo_status_box_pinned"
type="button" class="btn btn-mini btn-success help-tooltip" style="display:none"
onclick="pinFooStatusBox(false)">
<i class="fa fa-thumb-tack fa-xs"></i>
</button>
</div>
<div style="position:absolute;top:5px;left:2px;text-align:left;font-size:18px;padding:0">
Foo
</div>
<div id="foo_settings">
<div id="foo_min_status" style="position:absolute;top:125px;left:2px;text-align:left;font-size:15px;padding:0">Min foo:</div>
<div id="foo_max_status" style="position:absolute;top:145px;left:2px;text-align:left;font-size:15px;padding:0">Max foo:</div>
<div id="foo_auto_status" style="position:absolute;top:165px;left:2px;text-align:left;font-size:15px;padding:0">Auto-foo:</div>
</div>
</div>
<script>
function fooAddStatusBox() {
document.getElementById('foo_status_box').onmousedown = function () {
fooStatusBoxDragStart();
return true; // Not needed, as long as you don't return false
};
};
</script>
The only action taken by fooAddStatusBox() is to register a callback for when the user starts to move the custom status box around the screen. Causing the custom status box is done by custom_start_monitor_handler() and when a timeout occurs with no mouse movement, then custom_timeout_handler() will hide the custom status box.
Saving the custom configuration
When the settings modal is closed, custom_setting_modal_closed_handler() is invoked.
<script>
function custom_setting_modal_closed_handler() {
if (fooConfigurationUpdateNeeded) {
console.log("user change foo settings, saving");
fooUpdateConfiguration();
}
}
</script>
fooUpdateConfiguration() gets the values from the custom tab in the setting modal, updates the status box (via fooUpdateStatusBox()) and invokes fooSetConfiguration().
<script>
function fooUpdateConfiguration() {
fooConfiguration = {};
fooConfiguration['min'] = document.getElementById("foo_min").value;
fooConfiguration['max'] = document.getElementById("foo_max").value;
fooConfiguration['auto'] = fooAutoSetting;
console.log("Update configuration:", fooConfiguration);
fooConfigurationUpdateNeeded = false;
fooUpdateStatusBox(fooConfiguration);
changing_settings = true; // XXXX keeps camera javascript logic from doing browser refresh due to foo settings changing
fooSetConfiguration(fooConfiguration);
}
</script>
fooSetConfiguration() sends the updated values to the camera via a custom URL.
<script>
function fooSetConfiguration(config) {
$.ajax({
type: "POST",
url: "/foo_set_configuration",
data: JSON.stringify(config),
contentType: "application/json; charset=utf-8",
dataType: "json",
async: true,
success: function(data) {
if (null != data) {
console.log("New configuration saved");
}
},
error: function(data) {
console.log("Save configuration failed");
}
});
}
</script>
Summary of custom JavaScript functions
| custom_refresh_cache_handler() | Called when webUI thinks it is talking to a different camera or a camera that has been power cycled. Occurs when camstatus configuration change count changes. |
| custom_timeout_handler() | Called when there is no mouse movement for the timeout duration. Used to hide the custom status box. |
| custom_camstatus_handler() | Called once a second when the global variable cached_camstatus has been updated. Polling is disabled when status box is displayed . |
| custom_status_box_handler(show) | Used to show or hide the custom status box. |
| custom_start_monitor_handler() | Called when the periodic monitoring of the camera starts. This is typically after the settings modal is closed. |
| custom_stop_monitor_handler() | Called when the periodic monitoring of the camera stops. This is typically after the settings modal is opened. |
| custom_window_resize_handler() | Called when the user changes the browser window size. Used to reposition the customer status box so it doesn't go off screen. |
| custom_window_onload_handler() | Called after the webUI is initialized so the custom UI can be initialized. |
| custom_setting_modal_closed_handler() | Called when the setting modal is dismissed. Intended to allow for a check to see if the user changed any custom settings, and if so, to send the updated settings to the camera. |
| custom_show_or_drag_menu() | Called when the user is dragging the a status box around. It might be the custom status box. |
| custom_stop_dragging() | Called when the user releases the mouse button after dragging a status box around. |
Debugging hints
I regularly use 2 debugging tools: monitor camera's log file and monitor web browser's inspector console and network traffic.
webUI debugging
To get more debug output from the webUI code, in the web browser's inspector console type: enable_debug=true. When you are exchanging data with the camera, click on the network tab to verify you are sending what you expect and the camera is responding as expected.
Application extension python debugging
Now you are in for it. You thought you bought a nice bundled up camera that captures amazing videos (well, you did). But the edgertronic camera is also customizable, so lots of development tools ship with the camera.
The edgertrnoic camera runs linux and uses busybox. If you are familiar with the posix command line tools (cd, ls, cat, less, cp, mount etc), you will find many of them available after you telnet into the camera.
The example app_ext_foo.py allows optional debug output to be enabled. You can restart the camera code with the debug enabled by telnet-ing into the camera and running
FOO_DEBUG=1 ws ; log # cntl-C to exit monitoring the /var/log/messages file using the helper script log
or to run the python code in the foreground
FOO_DEBUG=1 app # cntl-C to exit the python code, you might type it several times as several threads are running
Running the python code in the foreground returns much more useful debug information when a python error is encountered.
Python debugging suggestions
- When you first get started, copy app_ext_foo.py and app_ext_foo.html to new files. They can be found in then camera's file system directory at /home/root/ss-web/static/sdk/app_ext/. I use app_ext_udpmon.py and app_ext_udpmon.html. In both files search and replace foo with your app extension name. Copy (or create NFS symbolic links as described below) to the camera's read/write file system directory /mnt/rw/etc/app_ext/. Run python in the foreground and verify your URLs get registered, you can read and write your configuration file (check contents using telnet), and the webUI extensions are shown properly.
- Now try all the other webUI extensions, which can be found in the camera's file system directory at /home/root/ss-web/static/sdk/app_ext/. Hopefully one of them does something close to what you are trying to do so you can see how it is done.
- You can list the loaded URLs by browsing to http://10.11.12.13/ext .
- When you first telnet into the camera, the lighttpd webserver is running as a daemon. The stop script will terminate lighttpd and the associated python process.
- Unfortunately, once a URL is registered with flask, I haven't found a way to unregister it. After you change your application extension python code, you have to restart the python code in order to register the URL again. If your application extension files are in /mnt/rw/etc/app_ext/ directory, then simply restarting python (via ws or app scripts) will automatically reload your extension. No need to put the extension on the big SD card during development.
- The rw script changes the linux root file system to be writable. You are now juggling sharp knives. You can break a million little details that will effect camera operation. If you contact us for support, please let us know you may have modified the root file system. We would appreciate it if you grabbed another microSD card, put on a clean image with a known good root file system and reproduce the issue using a unmodified root file system before contacting us.
- In general, you can restart the python code many times before you have to do a full reboot. If you happen to stop the python code such that the video processing pipeline gets unhappy, then you have to reboot. You can use the reboot script to reboot.
- Remember to refer to the CAMAPI documentation.
- I have included the source code to /home/root/ss-web/static/sdk/camera/app.py, which is the core camera functionality handling both loading application extensions and exposing CAMAPI via JSON over HTTP GET and POST. You can see example what the camera is doing when loading your extension. You can monitor in depth app's behavior by restarting the python code using APP_DEBUG=1 app .
Developing with NFS
Anytime I am developing python code, I NFS mount the files. This allows me to use my desktop source code editor and manage changes with git. You will need to figure out how to expose your desktop file system using an NFS server. I have done this many time with a Linux desktop and a Mac desktop. I haven't used Windows at work for development for 25 years.
As I developed udpmon application extension for a customer, I captured how I setup NFS and associated file symbolic links on the camera.
rw mkdir -p /mnt/nfs mount -o remount,noatime,nodiratime,ro /dev/root / # dang, I need to make a ro script to remount read-only mount -t nfs -n -o nolock,rsize=1024,wsize=1024 10.111.0.8:/Users/tfischer/work /mnt/nfs # change to your desktop's IP address and NFS share directory cd /mnt/rw/etc/app_ext SRCDIR=ss-clean/myapps/ss-web/extras_overlay/home/root/ss-web/static/sdk/app_ext # my crazy path to where I store app ext in the git repo ln -s /mnt/nfs/$SRCDIR/app_ext_udpmon.py app_ext_udpmon.py ln -s /mnt/nfs/$SRCDIR/app_ext_udpmon.html app_ext_udpmon.html UDPMON_DEBUG=1 APP_DEBUG=1 app
Watch the telnet console output for errors. Then browse to http://10.11.12.13/ext to verify your URLs were properly added.
Rebooting the camera causes the NFS mount to be lost. You can put the above lines in a new file named /etc/rc.d/S50nfsmount (first setting the camera's root file system to read/write and make the file executable). The mount will then happen at each boot during the sysV style initialization supported by busybox. I recall needed a sleep 10 before the mount, but I don't recall why.
| ← |
⌂ Home |
→ Extending or replacing the web UI |