News – UK Drone Show 2016

New Drones

DJI

The DJI stand held two of their new models.

The Mavic Pro is a portable system with collapsible arms allowing it to fit into a small backpack. Its’ FlightAutonomy technology allows obstacle avoidance and hover precision, while ActiveTrack allows the drone to follow the subject matter with a number of different shooting modes. It comes with a 3-axis gimbal and 4K camera.

DJI Mavic Pro

It is available for pre-order for £1,099.

The Phantom 4 Pro is an upgrade of the Phantom 4 . It improves on a number of areas of the previous drone:

• Improved camera with a 1-inch 20 megapixel sensor from a 1/2.3 inch 12.3 megapixel sensor on the Phantom 4.
• Stereo vision sensors on the rear of the drone in addition to the front facing ones that were on the original.
• New infrared sensors on the left and right of the drone.

DJI Phantom 4 Pro

The Phantom 4 Pro costs £1,589, while a version where the remote controller has an integrated screen costs £1,819.

Yuneec

A new combined thermal and RGB camera for the Typhoon H drone was available.

CGO-ET Dual Thermal/RGB Camera

The camera costs £1,799.

New Technologies

A number of technologies under development were on view in the Innovation Zone.

Tetradrone

An interesting concept of combined drone and submersible, changing the type of propeller allows the drone to either fly or go underwater with the body being watertight.

www.tetradrones.co.uk

http://jgarnham94.wixsite.com/tetra

https://www.facebook.com/TetraDrones/?hc_ref=SEARCH

Tetra Drone

Available for funding soon on Kickstarter.

Seadrone

An innovative modular underwater drone by van Dijk FEM engineering B.V. with a camera and a crab module for removing material from the sea bed.

http://seadrone.nl/

https://www.facebook.com/SeaDroneNL

SeaDrone

Droneball

Another innovative technology was droneball, which was a drone fully enclosed in a cage stabilizing system which protects the drone and its surroundings from damage as it flies around, it either fly or roll along the ground.

DroneBall

The droneball is getting launched on indiegogo on the 8th of December.

https://www.indiegogo.com/projects/droneball-the-bouncing-crash-resistant-drone-drones/coming_soon

 

 

News – Flyt

The uses of computer vision technologies in controlling UAV flight have been mentioned in some blogs already and will be discussed in more detail in a future blog. It can provide autonomous means of flight which are difficult to replicate manually allowing the use of a UAV in agriculture, inspections, surveys, delivery or emergency response.

The means to implement computer vision technologies on UAVs has generally meant proprietary technology or a complicated open source route with the setup of hardware and the installation of a number of pieces of software before experimentation can even begin.

But the work of  Indian Navstick Labs has developed a number of products to solve these problems.

FlyTOS

FlyTOS operating system is an application development framework built upon Linux and ROS (Robot Operating System), meaning an integration with ROS modules/libraries and sensors. It also supports the APM and PX4 (Pixhawk) open source autopilot systems.

The systems allows the development of obstacle avoidance, autonomous landing with AR tags and object recognition, tracking and following. It’s object tracking can use simple OpenCV based algorithms to detect objects using color and shape and use a Kalman Filter for tracking. It can also incorporate OpenTLD for selecting objects in a display and then following them (this was originally published in MATLAB by Zdenek Kalal).

FlytConsole and FlytVision are inbuilt on-board web apps that aid on in the creation of applications.

It comes with  a web-based control station called FlyConsole and a 3D simulator called FlytSim.

The software can be downloaded for free and installed on an ODROID XU4 companion computer.

ODROID XU4

FlytPOD

The FlytPOD – Advanced Flight Computer is a companion computer system running the FlytOS system which is currently being funded as part of an Indiegogo project.

FlytPOD

As well as coming with a suspended IMU for vibration damping and an external magnetometer it also supports RTK (Real Time Kinematics) GPS.

It’s USB3.0,  USB2.0, HDMI and user configurable I/Os connectors support a number of systems out-of-the-box including a Gimbal, PX4Flow (Optical Flow sensor), LiDAR (distance sensor) and USB Cameras. While the hardware interfaces on the FlytPOD support a number of specialized sensors including multi-spectral cameras, stereo cameras and LiDAR.

It is designed to be able to process photographs in the companion computer and stream them to ground.

The system comes in two models:

1. FlytPOD Kit – uSD storage. It costs $499 ($399 in Indiegogo).
2. FlytPOD PRO Kit –  This kit has the same features as the basic one but also offers sensor redundancy, with triple 3-axis Accelerometers and 3-axis Gyroscopes as well as double external Magnetometers, Barometers and External GPS. It also comes with the faster eMMC storage. It cost $799 ($699 on Indiegogo).

The Indiegogo funding ends on the 3rd October.

Increasing UAV Flight Time

One of the main problems with the use of UAV to record archaeology and cultural heritage is the amount of time that they can stay in the air. This is linked to the limitations of the current LiPo (Lithium-ion polymer) battery technology.

The increasing use of sensors and companion computers on-board also adds to the drain on batteries during flight.

In many cases it is solved by simply having a number of batteries, but of course this is costly with many of the smart batteries for commercial UAVs costing over £100 each. Larger systems can have up to 6 batteries on them allowing extended flight times but costing circa $1,200. With smaller systems this means that the survey needs to stop and start while the UAV lands and takes off adding time to a survey and requiring overlapping of the survey so that nothing is missed.

There are a number of developing technologies that can aid in this problem.

Battery Improvements

A number of developments in battery technology are being made driven by the technology industry, car industry and Formula E Racing that improve on the Lithium-ion technology that is currently employed.

Developments at the Pohang University of Science and Technology in South Korea could double the power of current Lithium-ion technology by combining porous stainless steel with thin-film electrolyte and electrodes. This would allow drones to fly for over an hour.

Battery Technologies (Extremetech.com)

Among the new battery technologies being researched are Zinc-air and Lithium-air batteries which have a much greater energy density than Lithium-ion batteries. Lithium-air batteries use oxygen as the oxidiser, and will potentially be 1/5 of the price 4/5 of the weight and have 5 times more more charge than Lithium-ion batteries.

Aluminium- air batteries have allowed an electric car to drive 1,100 miles on a single charge (developed by the Phinergy company), because the metal is turned into aluminium hydroxide as the battery is drained the battery needs to be recycled after use.

Cables and recharge stations

Cables are one possibility that can potential provide limitless flight time, they could provide an ideal source of power for localised recording but are limited by the length of the cable which would limit mapping missions.

In May at the  NAB 2016 conference 3DR announced the release of the Hoverfly Tether for their Solo drone which has a 150 ft-long cable. The system plugs into standard electrical outlets.

3DR Hoverfly Tether for Solo drone

For recording at a distance from the nearest power source a recharge station such as the DRONEBOX by HUS Unmanned Systems can be used, it provides a mobile generator for wireless charging a drone on a landing pad. While the REMOBOX is also available which is a tethered version of the system.

Dronebox

Hydrogen fuel cells

While their sister company, HES Energy Systems, is involved in the introduction of hydrogen fuel cells into UAV flight.

They can provide a significant saving in weight and increase in flight time, by increasing flight time to hours rather than minutes.

Battery vs. Hydrogen Fuel Cell (HES Energy Systems)

A number of manufacturers are developing the technology including the UK Intelligent Energy company whose work was discussed on an episode of BBCs Click.

Intelligent Energy Drone

In April the drone manufacturer MMC announced the first commercially available hydrogen-fueled drone – HyDrone 1800 – with a flight time of over 3 hours and a flight radius of 100 km.

HyDrone 1800

Using existing mapping data to control UAV mapping flights – Part 1 – Preliminary Ideas and Experimentation

An intrinsic problem with photogrammetry is its requirement to keep the camera facing the subject matter. A much higher quality and more accurate 3D model is produced using the method than taking photographs at an oblique angle. This is especially true of buildings with with flat facades, (this has already been discussed in another blog).

Work has been done using computer vision to automate the control of the camera position so that it follows targets selected by the pilot. Although this has potential for some recording methods such as site tours, as discussed in another blog, it doesn’t aid in the recording of complex topography or architecture. Although there is potential for the recording of architectural elements using computer vision  technologies (this will be discussed in a later blog).

Other work is being done in using a low detail 3D model of a building to aid in the control of a UAV flying around it, but these are more aimed at collision avoidance than quality recording.

While in the future i plan to look at the potential pre-scanning a building with an aerial LiDAR scanner mounted on a drone before recording with UAV.

Potential solution

The camera gimbal of a UAV can be controlled both remotely and from the autopilot of the UAV which could be used to always keep the camera facing the subject matter, but without pertinent information this would have to be done manually. With wireless camera technology it is possible to remotely view what the camera is recording and so control the movement of the gimbal when required, but this would require a second person to control the camera while the UAV is being flown and would be difficult to implement effectively and costly in a commercial environment.

But it would seem to be possible to use existing 3D data of an area to control the flight of a UAV; both controlling the altitude and the angle that the camera gimbal is pointing. I have already discussed the use of DroneKit Python to create a UAV mapping flight, thish can also be used to control the angle of the camera gimbal.

Existing Data

There are a number of existing sources of data that can be used to aid in creating a mapping flight.

Within the UK LiDAR data is freely available at different spatial resolutions, much of the country is available down to 1m while other areas are available down to 0.25 m.

This resource through processing in GIS (Geographic Information System) software provides all of the information required to create a flight path over the area under study and to control the angle of the camera gimbal so that it will record it to a higher quality than before.

A digital elevation model (DEM) created using photogrammetry from existing overlapping aerial photographs can also be employed once it is georeferenced to its correct location. This resource may provide a higher spatial resolution than the LiDAR data and so a better resource for the creation of the flight path, but the landscape and structures may have changed since the photographs were taken causing problems (this can of course be a problem with the LiDAR data as well).

Co-ordinate system problems

One complication with using LiDAR data to control the UAV is the fact that it is in a different co-ordinate system than the GPS of the UAV (OSGB and WGS84). This can be solved be translating one set of data to the co-ordinate system of the other. As the number of points for the mission path will be a lot less than that for the LiDAR data it would make sense to convert the GPS data to OSGB, but this also requires that it be converted back after the flight path has been created added a certain amount of inaccuracy into the data as a conversion is never 100% accurate.

required Data

Three different pieces of data need to be derived from the LiDAR data which are required for the UAV mapping flight:

• Altitude.
• Slope.
• Aspect.

The Altitude is contained within each point of the LiDAR data and is used when displaying the data in GIS software.

The Slope of the topography/buildings is measure in increments up to 90 degrees, with o degrees being flat and 90 degrees being a vertical face.

The Aspect is which way any slope is pointing in is measured in increments from 1-360 degrees. (degrees).

 

Slope angles

Although it would be possible to create software that extracts the data from the LiDAR file while creating a flight path this is not currently an option. The flight path is currently created in a piece of software such as the Open Source ‘Mission Planner’ system. In this an area is chosen together with other variables and an optimal flight path is created. This flight path file can then be saved, it contains the X and Y co-ordinates of each point of the mission.

UAV Control

At its simplest the flight path can be created with the altitude and slope derived from the LiDAR being used to control both the UAV altitude and camera gimbal angle. This would work well for sloping topography but would be more complicated for areas with sharp breaks in slope (such as buildings).

Altitude Control

The altitude will need to be carefully controlled to make sure that the quality of the imaginary is consistent across the whole area under study. At its simplest this is easy to do using the altitude data within the LiDAR data, together with obstacle avoidance sensors to aid with safety.

The problem arises when needing to record something near or completely vertical. Rather than requiring a set altitude the UAV needs to maintain a set distance horizontally. This may be possible by creating a buffer in the data around steeply sloping areas.

Problem with vertical offset

Camera Gimbal Control

Most low cost UAV systems come with a 2-axis gimbals, this means that the camera is stabilized so that it always stays in a horizontal plane but also that its rotation can be controlled downwards.

The angle of the gimbal begins at 0 degrees for a forward pointing position to 90 degrees for a downwards facing position. This is how is its controlled within DroneKit.

As seen earlier the slope is calculated between 0 and 90 degrees for a slope.

There are two intrinsic problems with this method:

1. The slope only goes between 0 and 90 degrees so there is no aspect data within it. If the drone camera is to be controlled to record the building as it flies over if needs to know which way the building is pointing as the 45 degrees on the left is not the same as the 45 degrees on the right. This could be solved by combining the information from the slope and aspect to give more detailed resulting data.
2. Most standard gimbals are designed to only point forwards and downwards. This means that the UAV has to turn around to record the back side of the building or it needs to fly the path in reverse. The other solution is to use a UAV with a camera that can point in 360 degrees.

GIS Processing

A certain amount of processing is required within GIS software to get the required data from the LiDAR data and combine it with the required mapping flight path. For this ArcGIS has been used both due to its availability at university and my own familiarity with it.

Lidar

Considering the LiDAR data is for a specific square it makes sense to use raster data rather than the points and lines of vector data as it retains the accuracy of the data. The LiDAR data can be simply loaded into the GIS software as a raster.

Within GIS software the Aspect and Slope can be calculated and a raster created showing the results.

This can be done using the Spatial Analyst or 3D Analyst Toolboxes to provide Slope and Aspect rasters.

The data for these can be incorporated into Attribute Tables which can be exported into text files. It is possible to combine all of the data into one attribute table containing the Altitude, Slope and Aspect.

Although it is possible to export this whole raster file including all of the data it is not currently possible to automatically derive the data in software using the flight path, so a flight path has to be loaded into the GIS software.

Flight Path

The flight path file we created in the Mission Planner software needs to be loaded into a feature class in the GIS software. This can be done by loading the point data for the flight path into the software, this is the beginning and end point for each of the back and forth paths across the area needed to be recorded.

We next need to recreate the flight path using ‘point to line’.

Even though we have recreated the lines, deriving enough data from them is not possible as the flight path is designed to fly back and forth at a set altitude. For this reason we need to create a number of extra points. This can be done using ‘Construct points’ where points can be created at set intervals along a line. This can be linked to the level of data that is being used, so for this LiDAR data the points can be set at intervals of 1m.

Once this his has been done ‘Extract multi-values to points’ can be run on the 3 sets of data to create a table containing all of the required data for each point on the flight path we have created.

UAV Mission Creation

Now that we have all of the required input data for the UAV mapping flight we need to create the mission within Dronekit Python.

For the first level of experimentation we can just load the point data file into python then create a number of points for the UAV to fly to which give the X and Y co-ordinates and the required altitude. At the same time we can also program in the angle for the camera gimbal. It may be best to have the UAV hover at the positions for a second or two so that we know how the recording is going.

As already mentioned if we are only using a 2-axis gimbal we are going to have to have the UAV turn through 180 degrees to record the back sides of buildings and slopes sloping away from the camera. We should be able to do this by altering the UAV Yaw. We will need to have the Python read the aspect angle and change how it creates the flight path depending on the aspect of the slope/building.

Future Directions

ArcGIS allows the use of Python to run tools in its toolbox so it seems possible to create a python script which would automatically create a file with all of the information required from input files of LiDAR data and a flight path.

As QGIS also allows the use of python it would also seem possible to create the required file within this open-source solution.

 

UAV Building Facade Recording – Part 1 – Preliminary Ideas and Experimentation

The recording of buildings is an important area in Cultural Heritage, whether for conditional surveys or to record something that is about to be destroyed.

Traditional methods rely upon survey equipment such as Total Stations to take a number of points on the façade, but this results in only points and lines with no great surface detail.

Other more detailed survey techniques such as laser scanning and photogrammetry have also been employed. But laser scanning is expensive and both the techniques are generally ground based missing detail of the façade that is not visible from this position. Scaffolding or a cherry picker can be used to record the whole of the building but again this can add to the cost the recording.

Photogrammetry is a low cost method of producing high quality results but relies upon having the camera parallel to the building to produce the best results, as capturing photographs from an angle brings inaccuracies into the recording as well as there being more detail at the bottom of the 3D model created than at the top.

The UAV would seem to provide an ideal platform to carry a camera parallel to the building, recording photographs with the required photogrammetry overlap. And with its autopilot it would seem possible to automate the recording process allowing the mapping of the façade in the same way that the UAV  can map the ground.

There are of course a number of problems that need to be overcome.

Building Façade Recording

Manual

Building façade recording can be done manually with a UAV, but the larger and more complicated the building façade the mode difficult it is to do this accurately. As the pilot needs to control the UAV accurately in 3 dimensions as well as controlling its speed.

Although the results for an experimental UAV mission are acceptable the difficulty of maintaining a manual position can be seen in the image below.

Bedham Mission Church – UAV Photogrammetry Model

Bedham Mission Church – UAV Camera Recording Positions

Automatic

In order to automate the process you need to determine what parameters are required to record a building façade using photogrammetry.

These can be seen below.

Building facade recording parameters

First experimentation was done by taking the co-ordinates of the two ends of an example the wall from Google Earth (The south facing wall of the lay brothers’ quarters at Waverley Abbey in Surrey was used). These co-ordinates can then be used to determine the bearing that the wall lies upon and its width. Using the camera parameters and level of detail the required distance from the wall for the flight can be calculated using trigonometry. Trigonometry is once again used to calculate the offset positions for the left and right extent of the flight.

 

Trigonometry used for calculations

The image overlap can be used to determine the number of photographs required in the horizontal and vertical, and hence the change of altitude that is required for each flight pass of the building.

Calculate altitude

Although it is planned to have the ability for the UAV to hover and take photographs, it is much easier to have it take photographs as it flies across the building façade. This requires the additional calculations and control of optimum flight speed and shutter speed to take photographs which are not adversely effected by motion blur.

Shutter speed formula

Shutter speed calculations

These preliminary calculations were done in Microsoft Excel.

DroneKit

The drone manufacturer 3DR provides a series of software development kits (SDKs) for writing applications to control your UAV using one of the open-source autopilot systems they support.

DroneKit Python uses the Python programming language and provides a number of examples to help with programming the flight of a UAV; these include flying from co-ordinate to co-ordinate up to complete missions. Together with this there is an API (application program interface) reference which provides all of the Python commands that can be used to control the UAV.

Python

Python is a fairly easy to learn programming language and as DroneKit already requires it to be installed and setup it makes sense to use the same language to calcuate the required paramaters for the flight path. This was done with the aid of a number of online resources. A graphical user interface (GUI) was created using the Tkinter Python package and was used to enter the data. The python code did the calculations then a file is exported which combines these calculations with the DroneKit code for controlling the autopilot. The final file when run will control the UAV flight.

Python GUI

Virtual Drone

Experimentation doesn’t need to be done with a live UAV, it can actually be done with a virtual one using a number of pieces of open-source software. These include Mission  Planner, ArduCopter, MAVProxy and SITL (Software in the loop)

Virtual Drone

Next Steps

Experimentation with a UAV using the hardware and software is the next step to test whether a GPS can be used in close proximity to a structure.

Limitations of standard UAV GPS accuracy to within the range of meters also complicates the use of this method of controlling the flight. This either needs to be solved with the use of a more accurate GPS (although the proximity to the building may block the signal), sensors that measure distances or the use of computer vision technologies to control the UAV position. The UAV afterall currently only need to fly between two set points then at set altitudes above the ground.

DJI Phantom 4

The DJI Phantom 4 is the new model in the popular phantom range of quadcopters, it has a number of improvements over previous models.

DJI Phantom 4

Comparison of DJI Phantom 4 and 3

Model	Phantom 4 Aircraft	Phantom 3 Professional or Advanced Aircraft
Battery	4S 15.2V 5350mAh Intelligent Flight Battery	4S 15.2V 4480mAh Intelligent Flight Battery
Max Flight Time	28 mins	About 23 mins
Vision Positioning System	10m	3m
Obstacle Sensing System	Optical Sensor – 0.7 – 15m	N/A
Intelligent Flight Modes	Follow Me Point of Interest Waypoints Course Lock Home Lock ActiveTrack TapFly	Follow Me Point of Interest Waypoints Course Lock Home Lock

Using the TapFly mode you can tap on a position of the screen in the app to fly to that location.

One of its main improvements is the introduction of obstacle avoidance technology (Sense and Avoid) using cameras mounted above the legs on the front of the Phantom 4.

DJI Phantom 4 – Obstacle Avoidance

The system, and the subsequent technologies, rely on a companion computer within the drone attached to the various sensors which uses computer vision algorithms to detect obstacles in the drones path. Once it has detected an obstacle it will either hover or fly around it.

DJI Phantom 4 – Companion Computer

It also comes with an improved Vision Positioning System, for position hold without the aid of GPS, which raises the positioning altitude from 3m to up to 10m.

DJI Phanton 4 – Vision Positioning

A final important new technology is ActiveTrack where a subject can be selected in the app, and once again using computer vision technologies, the Phantom 4 will follow the subject even when it is turning.

DJI Phantom 4 – Active Track

The DJI Phantom 4 is available for £1,229.00 and will be on general release from the 23rd of March. As such it will be the first commercially available drone with obstacle avoidance technology.

Benefits

The Phantom 4 provides a number of cutting edge technologies on a low cost platform. The benefit of ActiveTrack has already been discussed in a previous blog – UAVs for site tour recording – Part 1 – Theory while the potential of the sense and avoid and vision positioning system technologies will be discussed in a future blog on building recording.

Drawbacks

The main drawback of the system is the fact that the camera is not of the same quality as the Zenmsue X5 which is available for the DJ Inspire 1 Pro/Raw. But even this camera is not of the same specifications as many standard DSLR or mirrorless cameras, only providing 16MP.

News – 3DR and Sony UMC-R10C

3DR have announced the integration of the new Sony UMC-R10C Lens Style Camera into there Solo UAV platform. This will include a custom gimbal for the camera. It will replace the current GoPro camera with one of the quality of a DSLR camera capable of taking 20MP+ photographs.

The camera appears to be similar to the Sony ILCE-QX1 discussed in a previous blog – Mirror-less Cameras and UAVs. Although the technology had great potential for use in UAVs due to the fact it does not have the body and weight of a normal camera, it had the serious limitation of not having a full manual mode. Hopefully this will be remedied in the new model.

Sony UMC-R10C

The UMC-R10C is going to be released by 3DR as part of a complete mapping and processing solution called SiteScan.

It is not currently known whether the camera and gimbal will be available separately at present.

The camera will be unveiled at the NAB (National Association of Broadcasters) Show in Las Vegas in April.

While the package combined with SiteScan is expected to ship in June.

UAVs for site tour recording – Part 1 – Theory

Thanks to UAVs there is a growing potential for the provision of high quality visualizations of sites from the air for public consumption; whether as part of the requirement of many archaeology companies as charities, as part of planning policies to interact with the public, or the growing importance of crowdfunding archaeological excavations (DigVentures) which require interaction with their backers. UAVs can provide a means of providing this sort of imagery as part of an overall recording strategy. This includes the recording of site tours which can provide details of a sites which can easily be disseminated to the public.

At its simplest the UAV can provide an aerial element to the video of the site tour by flying past or through elements of the site or flying past or hovering in front of the site tour guide.

The DJI Inspire 1 is one such aerial video platform which can be purchased with two remote controllers; one for controlling the UAV, while the other is used to control the camera gimbal. This allows a pilot to fly the UAV on a set path while someone experienced in film making has complete control of the camera.

DJI Inspire 1

Although the UAV can provide an excellent platform for aerial video recording as part of site tours, recently developed technologies can make this much more automated and provide a means for one person to both:

1. The site tour guide.
2. The UAV pilot recording the site tour.

There are two ways in which this can be done.

1. GPS ‘Follow Me’ technology

‘Follow Me’ technology (DroneDog using Pixhawk)

This functionality is available on many UAVs, including some of the DJI series and those using the open source PX4 and Pixhawk autopilot technologies.

With the PX4/Pixhawk systems the mode can be controlled from a number of base station software solutions including Tower, which can run on Android mobile devices such as smartphones.

The systems uses the GPS of the mobile device as a target for the UAV.

A number of cinematic controls for the UAV are available in the app:

• Leash – UAV follows actor.
• Lead – UAV leads actor pointing back at them.
• Left/Right – UAV keeps pace with actor to the side.
• Circle – UAV circles actor at specified radius.

‘Follow Me’ controls (3DR Tower)

The following parameters can also be set:

• Altitude.
• Radius.

3DR Tower – Altitude and Radius

The system also controls the camera gimbal, pointing the camera towards the GPS enabled device.

Together these controls can provide various aerial video elements useful for integration in a site tour video which can be controlled directly from the mobile device in the hand of the site tour guide.

2.Computer vision technologies

Computer Vision technologies are an important developing area in robotics and are beginning to be fitted to UAVs.

Some of these technologies use image recognition algorithms to match the subject matter between consecutive video frames allowing the UAV to follow a person or object even when it is rotating and so changing the way it appears.

They come in three forms:

A. Software

Currently in beta testing the Vertical Studio app (available on iOS and Android) uses existing camera hardware on the DJI Phantom 3 or Inspire to provide the imagery for the image recognition algorithms running in the app. A target is chosen in the app which then controls the flight of the UAV.

Vertical Studio App

You can also draw walls in the app that designate no fly areas for the UAV.

Walls in the Vertical Studio App

B. Add-on technology

The second is an add-on technology that is fitted to an existing UAV, which connects to the autopilot and controls the flight of the UAV. In the case of the Percepto (funded on the Indiegogo crowdfunding website) the processing is done in a companion computer while the video is taken from an add-on camera, controls are then sent to the autopilot and gimbal to control the movement of them in relation to the subject matter.

Percepto Tracking

Percepto Kit

C. Integrated technology

The third is an an integral part a newly built UAV, but is in effect a very similar technology to B.

This is the case with the soon to be released DJI Phantom 4, which is the first commercially available UAV with the technology integrated into it.

DJI Phantom 4 – Obstacle Avoidance

DJI Phanton 4 – Vision Positioning

The app connects to a companion computer on the UAV which uses the imagery from the camera as a source for the computer vision algorithms. Once again the subject matter is selected in the app and the UAV will follow it.

Phantom 4 App

 

Sources
https://3dr.com/kb/follow-instructions/

http://www.dji.com/product/phantom-4

http://www.dji.com/product/intelligent-flight-modes

http://vertical.ai/features/

http://www.percepto.co/

Chris Anderson – InterDrone Keynote – The future of Drones

On the 9th of September Chris Anderson, the CEO of the 3DRobotics (3DR) Drone company, gave the keynote speech at the International Drone Conference and Exposition (InterDrone) Conference in Las Vegas on the future of drones.

Mapping
He discussed the previous democratizing of technologies such as the personal computer and the internet which put powerful tools in the hands of normal people empowering them; and that thanks to drone development remote sensing will develop in the same direction.

With improvement satellites have the potential to reduce their costs in remote sensing by a factor of 1 in 10, while drones have the potential to reduce the costs by a factor of 4 in 10.

The reason that the mobile phone network beat satellite phones is the same reason that drones will beat satellites in recording. Drones have the ability to scale in a way that satellites cannot.

He compared the pros and cons between Satellite, manned aircraft and drones in a hypothetical recording of the American State of Kansas. His model suggested that satellites and drones would have the same setup costs; with 100 satellites costing 100 million dollars and 200,000 drones costing the same amount to capture Kansas on a daily basis.

The satellites have high maintenance costs as they are located in space, while drones have much lower costs while piloted and less when un-piloted.

	Satellite	Drone
Data Resolution	300 cm per pixel	3 cm per pixel
Data Received	Hours	Minutes

The drones have the potential to record a site on the hour every hour.
Also at any one times 2/3 of the planet is under cloud cover making the drones the much better option as they generally fly under the clouds.

He concluded that for mapping 50 hectares the satellites would be the much cheaper option, but for mapping 5 hectares the manned drone is the cheaper option, while the manned aircraft was the worst option in all cases. But autonomous drones operating as a fleet would be the better option all the way up to 500 hectares.

Autonomous Drones
With the development of autonomy drones, rather than doing things in the same way as manned aircraft they could do things in whole new innovative ways.
They could:

• Act as unmanned sensors.
• Record every hour on the hour.
• Charge themselves on charging pads.
• Be based in weatherproof boxes.

These technologies are possible now and 3DR support both swarming and fleets.

It could be possible to control 200,000 drones from a web browser a thousand miles away. Both 3DR and DJI are migrating all of their control functions to mobile devices meaning that the platforms can act as connected devices from the start.

The development of the drone can be linked with sales:

• Drones as vehicles – 500 thousand.
• Drones as cameras – 1 million.
• Drones as something bigger – 10-100 million potential.

The drones have the potential to act as sensors connected to the internet almost forgetting about the device, with its data being passed to the cloud.

Potential drone uses
He discussed the potential of drones taking part in a number of the popular modern technological buzzwords:

• Internet of things.
• Big data.
• Reality capture.
• Personal storytelling.
• The democratisation of Hollywood.
• Video age.
• Personal robotics.
• Automation.
• The new near-space race.

He also discussed how drones came about thanks to mobile phone technology and its significant investment in the research and design of the technologies which enable phones to work including GPS, accelerometers as well as their computing technology. The mobile phone industry developed these technologies which allowed the drone industry to begin. The drone is basically a smartphone with propellers.

Technological/Future Developments
With improvements in the on-board computing processing power drones now have the ability to us the Robotic Operating System (ROS) allowing the drone industry to draw on the extensive academic developments in AI (Artificial Intelligence), Computer Vision, path-finding and other technologies. With ideas from a number of different industries converging.

The Dronecode project has already brought together a number of open source drone projects under the governance of The Linux Foundation.
These include:

• Ardupilot, PX4 and Pixhawk autopilot systems.
• MAVLink (Micro Aerial Vehicle Communication Protocol).
• UAVCAN is a lightweight protocol designed for reliable communication in aerospace and robotic applications via CAN bus.
• ROS – The Robot Operating System (ROS) is a set of software libraries and tools that help you build robot applications.

Cloud based systems such as droneshare and dronedeploy allow both the control and processing of drone data from within the cloud.

Other future directions could include:

• Advanced precision landing capabilities
• On-board HD video streaming support.
• Optical flow and GPS denied navigation.
• Standardized vision processing.
• Obstacle detection and avoidance (LIDAR, IR/thermal).
• Out-of-the-box Search and Rescue capabilities.

The drone has become possibly the most complex consumer product outside of the car industry, and continues to evolve.

High Dynamic Range Photography/Photogrammetry – Part 1

High Dynamic Range (HDR) Photography

High Dynamic Range is a popular photographic technique that is used to produce more realistic photographic results or artistic images. It is a technique that can be used to try and replicate what the human eye can see as the dynamic range of a camera is limited and it is unable to record the lightest and darkest elements in a single photograph. This can be remedied by taking a number of photographs with varying shutter speed/aperture combinations and combining them using specialist software to produce a photograph with a greater dynamic range than can be recorded in a single photograph.

Many new digital cameras have the ability to produce HDR photographs using Auto Exposure Bracketing (AEB), special HDR settings (this may process the images for you resulting in an HDR photo on the camera but losing the originals) or manually setting the camera up.

Limitations of archaeological and cultural heritage photography

An intrinsic problem with taking photographs in archaeological and cultural heritage contexts is lighting; both too much and too little lighting are factors that hamper recording images that include as much detail as possible.

In the case of archaeological excavations attempts have been made to limit the problems in section photography by either reflecting more light in using white card or using a tarpaulin to cast a shadow. Both of these techniques work but require time and manpower.

The same problem is encountered in building recording where in an outside environment strong lighting can cause both bleached out areas and heavy shadows.

Netley Abbey, Hampshire – East Window. Photograph demonstrating the problem of strong oblique lighting causing both too much and too little light in the same image.

While lighting through windows can cause similar problems on the inside of buildings.

Netley Abbey, Hampshire – Sacristy/Library. Photograph demonstrating the effect of excessive light coming through a window causing the dual problems of both too much light near to the window and too little light in other areas.

In order to reveal elements in dark shadow a high exposure camera setting is required, while bright bleached out areas are only revealed with low exposures. These elements together with well lit areas cannot by revealed in one single photograph, this is where High Dynamic Range photography comes in.

HDR Photography in Archaeology

HDR Photography was introduced to Archaeology by David Wheatley in 2010, he provided examples of its use in improving the standard recording methods of excavations, cave sites and even using archived analogue archaeological photographs. Sadly its use was not embraced by the community probably due to technological limitations at the time and the inherent conservatism of the industry and museum archives which were yet to embrace digital photographic technology. Technology and the industry have now caught up with his ideas, with digital cameras being present on most if not all excavations, while other scholars have now begun to bring the technology to the technique of 3D recording using photogrammetry.

The interior of one of the rock-cut tombs at the necropolis of Cala Morel (Menorca) (Wheatley 2010)

Test pit recorded during investigations at Itchen Abbas, Hampshire (Wheatley 2010)

HDR Field Archaeology Photography

Photography is one of the primary recording techniques within field archaeology and has been since the introduction of discipline, but conservatism within field archaeology has meant that it was only fairly recently that digital photography became the primary recording technology.

Digital cameras have a number of benefits within archaeology:

• The ability to take numerous photographs on one memory card.
• No need to pay to process films.
• No need to digitize the photographs.
• Where once excavators may have been told to limit the number of photographs taken on an excavation to keep the processing costs down, digital media allows almost limitless photographs to be taken.
• Photographs can be as easy as point and click with the camera controlling all of the settings.

But they also have drawbacks:

• Where once archaeologists knew how to use an analogue camera to take bracketed shots, the automatic setting on digital cameras is commonly the only setting used as it produces results at a required level of quality, this means that the archaeologist may not know how to properly operate the camera.
• Although almost limitless photographs can be taken, limits should be included as the archive may still need to be sorted through.
• The requirements for digital storage can be complicated and costly.

Although not ideal, a number of modern cameras now come with an HDR setting on them which in many cases can be changed to the required level of bracketing, although only the merged photograph is saved losing the possibility of later re-processing the photographs with different settings.

Field Archaeology Archive Photographs

One benefit of traditional bracketing of analogue photographs for archaeological excavations is that they provide an ideal resource for conducting HDR processing. These archives have multiple photographs at different exposure levels which can be digitized and processed to provide better results than the originals and be re-entered into the archive with the digitized originals.

HDR using archive slides from excavations at the Cove, Avebury (Wheatley 2010)

HDR Building Photography

Building recording is an area that can be significantly enhanced by the use of HDR. It is difficult to provide adequate lighting in many cases, meaning that some areas are brightly illuminated while others are dimly lit loosing information in both cases.

Processing bracketed images into an HDR image provides a greatly enhanced image.

Netley Abbey, Hampshire. East Window comparison – Standard Photograph

Netley Abbey, Hampshire – East window comparison – HDR Photograph

HDR Photogrammetry

Recent developments in camera technology, HDR software and photogrammetry software have allowed the introduction of HDR Photogrammetry. Thanks to the additional information present in the photographs models of higher detail and accuracy can be created in non-optimal lighting conditions.

As well as the ability to use tone mapped images produced from HDR images the Agisoft PhotoScan Photogrammetry software can also process .exr file format High Dynamic Range images into 3D models.

HDR Object Photogrammetry

One area under study is its use in photographing objects. The benefits are determined by the type of material used, some are greatly enhanced by HDR while others are little altered.

Image matching result from the images originated
with different HDR processing: a) No HDR; b) tone mapped
images from HDR processing (Guidi et al 2014)

HDR Building Photogrammetry

We have already seen the benefits of HDR Photography in building recording and this can continue with photogrammetry.

Photogrammetry point cloud of the east window of Netley Abbey, Hampshire showing how the raking sunlight on the left-hand side of the window has bleached out the photographs and lost detail

Both the increased level of quality of the photograph and the higher amount of detail present in the 3D model can easily be seen in the HDR photogrammetry model.

Standard Photogrammetry using standard .jpg photographs

HDR Photogrammetry using processed .exr files

Software Solutions

A number of software solutions are available for the processing of HDR photographs, these range from high end photographic software such as Adobe Photoshop and Lightroom, through to HDR specific pieces of software and even open source solutions. HDRSoft’s Photomatrix comes in a number of versions which include plugins for different software packages such as Adobe Lightroom, Photoshop Elements, Photoshop and Apple Aperture. With low cost solutions such as Fusion HDR or free open-source solutions such as LuminanceHDR also being available.

In order to be view-able on low contrast monitors and paper the images need to go through a process called tone mapping, this replicates the appearance of the high dynamic range photograph on these media.

Downloaded imaged can be batch processed in software such as Photomatrix setting up how many images need to be merged together with a number of preset or custom settings allowing the images to be processed exactly as required. These pieces of software can also compensate for slight movement between the recording of the multiple images. The resulting images can then be saved as either .hdr (Radiance) or .exr (OpenEXR) file formats which record the HDR information.

Batch processing of images within Photomatrix

Benefits
HDR photography can record more information in both photographs and photogrammetry models. By using open Source HDR software it can be free. Many cameras allow multiple bracketed photographs to be be taken automatically only adding a few seconds to the recording process.

It is also possible in some of the software to open a folder full of images and have the software batch process it without any user intervention once the preset settings have been loaded.

Drawbacks
Among the drawbacks are the fact that as the camera is taking multiple photographs it is difficult to stabilize the camera by hand, otherwise there will be movement between the photographs. Although movement between photographs can be corrected if you are bracketing shots and using software the automatic HDR setting on the camera will probably result in a blurry image.

UAV HDR Photogrammetry

UAV HDR Photogrammetry is an area I will be studying in the future. It has great potential for recording but will require a careful balance of UAV hovering, a steady gimbal, fast shutter speed and an adequate depth of field. It will be discussed in a future blog.

Sources
Guidi, G., S. Gonizzi, and L. L. Micoli. “Image pre-processing for optimizing automated photogrammetry performances.” ISPRS Annals of The Photogrammetry, Remote Sensing and Spatial Information Sciences 2.5 (2014): 145-152.

Kontogianni, G., and A. Georgopoulos. “Investigating the effect of HDR images for the 3D documentation of cultural heritage.” World Cultural Heritage Conference 2014 – Euromed 2014 – International Conference on Cultural Heritage Documentation, Preservation and Protection. (2014)

Ntregkaa, A., A. Georgopoulosa, and M. Santana Quinterob. “Photogrammetric Exploitation of HDR Images for Cultural Heritage Documentation.” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 5 (2013): W1.

Wheatley, D. “High dynamic range imaging for archaeological recording.” Journal of Archaeological Method and Theory 18, no. 3 (2011): 256-271.