Tag Archives: pixhawk

UAV Building Facade Recording – Part 1 – Preliminary Ideas and Experimentation

The recording of buildings is an important area in Cultural Heritage, whether for conditional surveys or to record something that is about to be destroyed.

Traditional methods rely upon survey equipment such as Total Stations to take a number of points on the façade, but this results in only points and lines with no great surface detail.

Other more detailed survey techniques such as laser scanning and photogrammetry have also been employed. But laser scanning is expensive and both the techniques are generally ground based missing detail of the façade that is not visible from this position. Scaffolding or a cherry picker can be used to record the whole of the building but again this can add to the cost the recording.

Photogrammetry is a low cost method of producing high quality results but relies upon having the camera parallel to the building to produce the best results, as capturing photographs from an angle brings inaccuracies into the recording as well as there being more detail at the bottom of the 3D model created than at the top.

The UAV would seem to provide an ideal platform to carry a camera parallel to the building, recording photographs with the required photogrammetry overlap. And with its autopilot it would seem possible to automate the recording process allowing the mapping of the façade in the same way that the UAV  can map the ground.

There are of course a number of problems that need to be overcome.

Building Façade Recording

Manual

Building façade recording can be done manually with a UAV, but the larger and more complicated the building façade the mode difficult it is to do this accurately. As the pilot needs to control the UAV accurately in 3 dimensions as well as controlling its speed.

Although the results for an experimental UAV mission are acceptable the difficulty of maintaining a manual position can be seen in the image below.

Automatic

In order to automate the process you need to determine what parameters are required to record a building façade using photogrammetry.

These can be seen below.

Excel Calculations

Building facade recording parameters

First experimentation was done by taking the co-ordinates of the two ends of an example the wall from Google Earth (The south facing wall of the lay brothers’ quarters at Waverley Abbey in Surrey was used). These co-ordinates can then be used to determine the bearing that the wall lies upon and its width. Using the camera parameters and level of detail the required distance from the wall for the flight can be calculated using trigonometry. Trigonometry is once again used to calculate the offset positions for the left and right extent of the flight.

 

The image overlap can be used to determine the number of photographs required in the horizontal and vertical, and hence the change of altitude that is required for each flight pass of the building.

Calculate altitude changes

Calculate altitude

Although it is planned to have the ability for the UAV to hover and take photographs, it is much easier to have it take photographs as it flies across the building façade. This requires the additional calculations and control of optimum flight speed and shutter speed to take photographs which are not adversely effected by motion blur.

Shutter speed formula

Shutter speed formula

Shutter speed calculions

Shutter speed calculations

These preliminary calculations were done in Microsoft Excel.

DroneKit

The drone manufacturer 3DR provides a series of software development kits (SDKs) for writing applications to control your UAV using one of the open-source autopilot systems they support.

DroneKit Python uses the Python programming language and provides a number of examples to help with programming the flight of a UAV; these include flying from co-ordinate to co-ordinate up to complete missions. Together with this there is an API (application program interface) reference which provides all of the Python commands that can be used to control the UAV.

Python

Python is a fairly easy to learn programming language and as DroneKit already requires it to be installed and setup it makes sense to use the same language to calcuate the required paramaters for the flight path. This was done with the aid of a number of online resources. A graphical user interface (GUI) was created using the Tkinter Python package and was used to enter the data. The python code did the calculations then a file is exported which combines these calculations with the DroneKit code for controlling the autopilot. The final file when run will control the UAV flight.

Python GUI

Python GUI

Virtual Drone

Experimentation doesn’t need to be done with a live UAV, it can actually be done with a virtual one using a number of pieces of open-source software. These include Mission  Planner, ArduCopterMAVProxy and SITL (Software in the loop)

Virtual Drone

Virtual Drone

Next Steps

Experimentation with a UAV using the hardware and software is the next step to test whether a GPS can be used in close proximity to a structure.

Limitations of standard UAV GPS accuracy to within the range of meters also complicates the use of this method of controlling the flight. This either needs to be solved with the use of a more accurate GPS (although the proximity to the building may block the signal), sensors that measure distances or the use of computer vision technologies to control the UAV position. The UAV afterall currently only need to fly between two set points then at set altitudes above the ground.

Advertisements

UAVs for site tour recording – Part 1 – Theory

Thanks to UAVs there is a growing potential for the provision of high quality visualizations of sites from the air for public consumption; whether as part of the requirement of many archaeology companies as charities, as part of planning policies to interact with the public, or the growing importance of crowdfunding archaeological excavations (DigVentures) which require interaction with their backers. UAVs can provide a means of providing this sort of imagery as part of an overall recording strategy. This includes the recording of site tours which can provide details of a sites which can easily be disseminated to the public.

At its simplest the UAV can provide an aerial element to the video of the site tour by flying past or through elements of the site or flying past or hovering in front of the site tour guide.

The DJI Inspire 1 is one such aerial video platform which can be purchased with two remote controllers; one for controlling the UAV, while the other is used to control the camera gimbal. This allows a pilot to fly the UAV on a set path while someone experienced in film making has complete control of the camera.

DJI Inspire 1

DJI Inspire 1

Although the UAV can provide an excellent platform for aerial video recording as part of site tours, recently developed technologies can make this much more automated and provide a means for one person to both:

  1. The site tour guide.
  2. The UAV pilot recording the site tour.

There are two ways in which this can be done.

1. GPS ‘Follow Me’ technology

'Follow Me' technology (DroneDog using Pixhawk)

‘Follow Me’ technology (DroneDog using Pixhawk)

This functionality is available on many UAVs, including some of the DJI series and those using the open source PX4 and Pixhawk autopilot technologies.

With the PX4/Pixhawk systems the mode can be controlled from a number of base station software solutions including Tower, which can run on Android mobile devices such as smartphones.

The systems uses the GPS of the mobile device as a target for the UAV.

A number of cinematic controls for the UAV are available in the app:

  • Leash – UAV follows actor.
  • Lead – UAV leads actor pointing back at them.
  • Left/Right – UAV keeps pace with actor to the side.
  • Circle – UAV circles actor at specified radius.
'Follow Me' controls (3DR Tower)

‘Follow Me’ controls (3DR Tower)

The following parameters can also be set:

  • Altitude.
  • Radius.
3DR Tower - Altitude and Radius

3DR Tower – Altitude and Radius

The system also controls the camera gimbal, pointing the camera towards the GPS enabled device.

Together these controls can provide various aerial video elements useful for integration in a site tour video which can be controlled directly from the mobile device in the hand of the site tour guide.

2.Computer vision technologies

Computer Vision technologies are an important developing area in robotics and are beginning to be fitted to UAVs.

Some of these technologies use image recognition algorithms to match the subject matter between consecutive video frames allowing the UAV to follow a person or object even when it is rotating and so changing the way it appears.

They come in three forms:

A. Software

Currently in beta testing the Vertical Studio app (available on iOS and Android) uses existing camera hardware on the DJI Phantom 3 or Inspire to provide the imagery for the image recognition algorithms running in the app. A target is chosen in the app which then controls the flight of the UAV.

Vertical Studio App

Vertical Studio App

You can also draw walls in the app that designate no fly areas for the UAV.

Walls in the Vertical Studio App

Walls in the Vertical Studio App

B. Add-on technology

The second is an add-on technology that is fitted to an existing UAV, which connects to the autopilot and controls the flight of the UAV. In the case of the Percepto (funded on the Indiegogo crowdfunding website) the processing is done in a companion computer while the video is taken from an add-on camera, controls are then sent to the autopilot and gimbal to control the movement of them in relation to the subject matter.

Percepto Tracking

Percepto Tracking

Percepto Kit

Percepto Kit

C. Integrated technology

The third is an an integral part a newly built UAV, but is in effect a very similar technology to B.

This is the case with the soon to be released DJI Phantom 4, which is the first commercially available UAV with the technology integrated into it.


The app connects to a companion computer on the UAV which uses the imagery from the camera as a source for the computer vision algorithms. Once again the subject matter is selected in the app and the UAV will follow it.

Phantom 4 App

Phantom 4 App

 

Sources
https://3dr.com/kb/follow-instructions/

http://www.dji.com/product/phantom-4

http://www.dji.com/product/intelligent-flight-modes

http://vertical.ai/features/

http://www.percepto.co/

3DRobotics Dronekit

3DRobotics have announced the release of DroneKit which offers an Open Source Software Development Kit (SDK) and web Application Program Interface (API) for developing drone apps. It works on systems powered by the APM flight code such as the ArduPilot, APM and Pixhawk autopilot systems, all supplied by 3DRobotics.

It allows the creation of custom purpose built UAV (Unmanned Aerial Vehicle) control apps without having to redesign the control system software.

The apps can be developed on three different platforms:

  1. Mobile apps with DroneKit Android.
  2. Web-based apps with DroneKit Cloud.
  3. Computer apps with DroneKit Python.

It enables the user to:

  • Control the flight path with waypoints.
  • Control a spline flight path with fine control over the vehicle velocity and position.
  • Set the UAV to follow a GPS target (Follow Me).
  • Control the camera and gimbal by setting Regions Of Interest (ROI) points which the camera locks on to.
  • Access full telemetry from the UAV using 3DR Radio, Bluetooth, Wi-Fi, or over the internet.
  • Playback and analyse the log of any mission.

The advantages of DroneKit are:

  • It is truly open unlike the similar DJI SDK, without levels of access.
  • Once an app has been created the interface is always the same across different computing platforms.
  • It can be used with planes, copters and rovers.
  • It works on laptop computers as well, mobile devices and vehicle data can even be accessed via the web.

DroneKit already powers a number of flight control programs:

  • The Tower (formerly Droidplanner) flight planning mobile app for Android was built on DroneKit for Android.

Tower (DroidPlanner 3)

  • Droneshare is a global social network for drone pilots that allows them to view and share missions, it is built on DroneKit web services.
  • Googles Project Tango Indoor Navigation is built on the Pixhawk and APM sutopilot systems and the Tower flight planning app.
  • The IMSI/Design TurboSite aerial reporting app for construction allows the setting up of flight waypoints to GPS locations and the capturing of photographs, videos, dictations, text notes and “punch list” action items. Photographs can be annotated while the UAV is still in flight using markup and measurements tools.

Aero-M

The Aero-M is one of two aerial mapping platforms introduced by 3D Robotics, it is a fixed wing aircraft . It is designed to have everything that you need for mapping straight out of the box and as well as having standard elements includes:

  • Pixhawk autopilot system.
  • Canon SX260 with custom 3DR EAI software and fixed mount.
  • Pix4Dmapper LT 3DR Edition, which is only capable of creating two-dimensional maps.
  • It also includes a custom-designed hard-top travel case.

Additional elements that can be purchased include:

  • An upgrade to Pix4Dmapper Pro 3DR Edition, this allows the creation and export of DSM (Digital surface models) and terrain models as well as orthomosaic editing. This costs an extra $5000.
  • An OSD (On screen display)/FPV (First person view) system which uses a Sony HAD 520 line camera to stream video to a viewing device while a MinimOSD on-screen-display module superimposes live telemetry data onto the video feed. This costs an additional $249.99. A monitor needs to be purchased to view the data.

It has a 40 minutes flight time and can record up to 250 acres in one flight.

http://store.3drobotics.com/products/aero-m/

The basic price of the Aero-M is $5400, with all of the additions it costs $10724.99. According to 3D Robotics this is providing “advanced mapping capabilities at a price five times less than that of our nearest technological competitor”..

Example
An example of an photo-mosaic map creating using the Pix4Dmapper software from an aerial platform. Map_Tease

Potential
The flight time of the Aero-M is significantly longer than the other mapping system, X8-M, which demonstrates the benefits of using a fixed wing platform over a multi-rotor platform which needs to power all of the rotors to stay in the air.

The multiple overlapping photographs taken by the camera can be used to create digital elevation models (DEM) these are an important element is archaeological prospection, although they can be created with LIDAR data a much higher level of detail can be generated from those created by photogrammetry using UAV (Unmanned Aerial Vehicle) photographs. The overalapping photographs can also be stitched together to create a high quality mosaic. They can also be used to create a 3D model of the site.

3d Robotics is part of the open-source hardware and software community based at DIY Drones which can help with all aspects of drone construction and use.

The Pixhawk autopilot system allows the easy creation of a flight path by selecting points on a map displayed in the mission planning software. By selecting a polygon around the area the software can create a grid flight pattern to fly; the altitude, camera type and overlap of images can also be set which alters the amount of times the aerial vehicle flies across the area under study.

Limitations
The problems with using a fixed wing platform for mapping are, takeoffs and landings, and tree cover around the area under study; the X8-M multi-rotor platform has the ability to turn on the spot.

Although the Aero-M is an expensive purchase it appears to still be a bargain.

The extra $5000 for software is also a lot, there are other cheaper options such as Agisoft Photoscan Professional which costs $3499.

It comes with a Canon SX260 camera which is a cheap, compact, lightweight, digital camera that fits well into the system; but it has neither the optics nor the image size (12.1 megapixels vs. 24.2 megapixels) of a higher quality digital SLR camera, so information will be lost from photographs that could be taken by a higher quality camera. The Aero-M platform can carry 500g so it will be able to carry higher specification digital SLR cameras than the Canon, allowing higher definiti0on pictures to be taken.

It is designed solely for mapping so the camera is in a fixed position and can only record downwards.

Although accuracy may have its limitations, by placing targets on the ground in a grid pattern and recording their position with GPS (Global Positioning System) the accuracy can be improved, and as the DEM is accurately georeferenced it can be imported accurately into GIS software.

X8-M

The X8-M is one of two aerial mapping platforms introduced by 3D Robotics, it is a multi-rotor UAV (Unmanned Aerial Vehicle) with a fully redundant propulsion system thanks to the fact that four of the propellers face up while the other four face down on the same struts, so if one fails there is still propulsion from the other. It has the ability to conduct mapping at a low altitude, at a low speed and at a high level of accuracy which is impossible to capture by manned flights or using satellite imaginary.

It is designed to have everything that you need for mapping straight out of the box and as well as having standard elements includes:

  • Pixhawk autopilot system.
  • Canon SX260 with custom 3DR EAI software and fixed mount.
  • Pix4Dmapper LT 3DR Edition, which is only capable of creating two-dimensional maps.
  • It also includes a custom-designed hard-top travel case.

Additional elements that can be purchased include:

  • An upgrade to Pix4Dmapper Pro 3DR Edition, this allows the creation and export of DSM (Digital surface models) and terrain models as well as orthomosaic editing. This costs an extra $5000.
  • An OSD (On screen display)/FPV (First person view) system which uses a Sony HAD 520 line camera to stream video to a viewing device while a MinimOSD on-screen-display module superimposes live telemetry data onto the video feed. This costs an additional $249.99. A monitor needs to be purchased to view the data.

The basic price of the x8-M is $5400, with all of the additions it costs $10729.98. According to 3D Robotics this is providing “advanced mapping capabilities at a price five times less than that of our nearest technological competitor”.

The X8-M has a flight time of 14 minutes covering 25 acres.

https://store.3drobotics.com/products/x8-m

Potential
The multiple overlapping photographs taken by the camera can be used to create digital elevation models (DEM) these are an important element is archaeological prospection, although they can be created with LIDAR data a much higher level of detail can be generated from those created by photogrammetry using UAV (Unmanned Aerial Vehicle) photographs. The overalapping photographs can also be stitched together to create a high quality mosaic. They can also be used to create a 3D model of the site.

3d Robotics is part of the open-source hardware and software community based at DIY Drones which can help with all aspects of drone construction and use.

The Pixhawk autopilot system allows the easy creation of a flight path by selecting points on a map displayed in the mission planning software. By selecting a polygon around the area the software can create a grid flight pattern to fly; the altitude, camera type and overlap of images can also be set which alters the amount of times the aerial vehicle flies across the area under study.

Example
An example of an photo-mosaic map creating using the Pix4Dmapper software from an aerial platform. Map_Tease

An example of this type of work with a multi-rotor UAV can be seen here.

While a 3D model created from images captured from a hexacapoter can be seen here on the p3d.in 3D model sharing website.

Limitations
Although the Aero-M is an expensive purchase it appears to still be a bargain.

The flight time of 14 minutes is quite small for a large area survey, but multiple batteries could be carried and mapping done in stages.

The extra $5000 for software is also a lot, there are other cheaper options such as Agisoft Photoscan Professional which costs $3499.

It comes with a Canon SX260 camera which is a cheap, compact, lightweight, digital camera that fits well into the system; but it has neither the optics nor the image size (12.1 megapixels vs. 24.2 megapixels) of a higher quality digital SLR camera, so information will be lost from photographs that could be taken by a higher quality camera. But the X8-M platform can only carry 200g so it is not capable of carrying these heavier cameras which weigh 530g (in the case of the Nikon D5300) so the quality is probably at the highest level possible.

It is deigned solely for mapping so the camera is in a fixed position and can only record downwards, a camera gimbal could be easily attached to the UAV frame allowing the camera to view a multiple angles and so record much more than just what is below it.

Although the accuracy of any DEM or photogrammetry model may have its limitations, by placing targets on the ground in a grid pattern and recording their position with GPS (Global Positioning System) the accuracy can be improved, and as the DEM is accurately georeferenced it can be imported accurately into GIS software.

News – Dronecode Project

The Open Source Dronecode Project has been announced under the auspices of the Linux Foundation, it will bring together existing projects including the APM/ArduPilot and PX4 open source autopilot systems as well as advancing new technologies. It will provide a common platform for Drone and robotics opens source projects aiming to unite the open source industry.

The maker community has already dramatically increased the development of drones and the Dronecode Project is hoping to advance the technologies required and both improve them and make them more affordable.

The Linux Foundation can provide an existing organisation and collaborative framework allowing the the Dronecolde Project to concentrate on the innovation of new technology.

http://www.linuxfoundation.org/news-media/announcements/2014/10/linux-foundation-and-leading-technology-companies-launch-open
https://www.dronecode.org/

News – 3D Robotics announces partnership with Intel

3D Robotics has announced a partnership with Intel in which they will be using the new Intel Edison for development of their autopilot systems. The Intel Edison is a microcomputer the size of a postage stamp which provides the power of a personal computer.

The extra processing power of the Edison will allow a person or object to be tracked with the follow me technology of the Pixhawk autopilot. So a person can be filmed automatically with the camera on the UAV (unmanned aerial vehicle) by tracking the person without the need for them to carry a mobile device, with its reliance on a less accurate GPS signal, as the UAV will be able to visually recognise a person.

It will also allow developments in image processing, sense and avoidance with new classes of sensors allowing further developments of autonomous UAV flight and object avoidance.