UAVs for site tour recording – Part 1 – Theory

Thanks to UAVs there is a growing potential for the provision of high quality visualizations of sites from the air for public consumption; whether as part of the requirement of many archaeology companies as charities, as part of planning policies to interact with the public, or the growing importance of crowdfunding archaeological excavations (DigVentures) which require interaction with their backers. UAVs can provide a means of providing this sort of imagery as part of an overall recording strategy. This includes the recording of site tours which can provide details of a sites which can easily be disseminated to the public.

At its simplest the UAV can provide an aerial element to the video of the site tour by flying past or through elements of the site or flying past or hovering in front of the site tour guide.

The DJI Inspire 1 is one such aerial video platform which can be purchased with two remote controllers; one for controlling the UAV, while the other is used to control the camera gimbal. This allows a pilot to fly the UAV on a set path while someone experienced in film making has complete control of the camera.

DJI Inspire 1

Although the UAV can provide an excellent platform for aerial video recording as part of site tours, recently developed technologies can make this much more automated and provide a means for one person to both:

1. The site tour guide.
2. The UAV pilot recording the site tour.

There are two ways in which this can be done.

1. GPS ‘Follow Me’ technology

‘Follow Me’ technology (DroneDog using Pixhawk)

This functionality is available on many UAVs, including some of the DJI series and those using the open source PX4 and Pixhawk autopilot technologies.

With the PX4/Pixhawk systems the mode can be controlled from a number of base station software solutions including Tower, which can run on Android mobile devices such as smartphones.

The systems uses the GPS of the mobile device as a target for the UAV.

A number of cinematic controls for the UAV are available in the app:

• Leash – UAV follows actor.
• Lead – UAV leads actor pointing back at them.
• Left/Right – UAV keeps pace with actor to the side.
• Circle – UAV circles actor at specified radius.

‘Follow Me’ controls (3DR Tower)

The following parameters can also be set:

• Altitude.
• Radius.

3DR Tower – Altitude and Radius

The system also controls the camera gimbal, pointing the camera towards the GPS enabled device.

Together these controls can provide various aerial video elements useful for integration in a site tour video which can be controlled directly from the mobile device in the hand of the site tour guide.

2.Computer vision technologies

Computer Vision technologies are an important developing area in robotics and are beginning to be fitted to UAVs.

Some of these technologies use image recognition algorithms to match the subject matter between consecutive video frames allowing the UAV to follow a person or object even when it is rotating and so changing the way it appears.

They come in three forms:

A. Software

Currently in beta testing the Vertical Studio app (available on iOS and Android) uses existing camera hardware on the DJI Phantom 3 or Inspire to provide the imagery for the image recognition algorithms running in the app. A target is chosen in the app which then controls the flight of the UAV.

Vertical Studio App

You can also draw walls in the app that designate no fly areas for the UAV.

Walls in the Vertical Studio App

B. Add-on technology

The second is an add-on technology that is fitted to an existing UAV, which connects to the autopilot and controls the flight of the UAV. In the case of the Percepto (funded on the Indiegogo crowdfunding website) the processing is done in a companion computer while the video is taken from an add-on camera, controls are then sent to the autopilot and gimbal to control the movement of them in relation to the subject matter.

Percepto Tracking

Percepto Kit

C. Integrated technology

The third is an an integral part a newly built UAV, but is in effect a very similar technology to B.

This is the case with the soon to be released DJI Phantom 4, which is the first commercially available UAV with the technology integrated into it.

DJI Phantom 4 – Obstacle Avoidance

DJI Phanton 4 – Vision Positioning

The app connects to a companion computer on the UAV which uses the imagery from the camera as a source for the computer vision algorithms. Once again the subject matter is selected in the app and the UAV will follow it.

Phantom 4 App

 

Sources
https://3dr.com/kb/follow-instructions/

http://www.dji.com/product/phantom-4

http://www.dji.com/product/intelligent-flight-modes

http://vertical.ai/features/

http://www.percepto.co/

Ghost Drone

The Ghost Drone is a wind and rain resistant Indiegogo project drone aimed at filmmakers, photographers, sports enthusiasts, travelers and adventurers, GoPro owners and first time and experienced drone pilots.

It is controlled by a smartphone app (either iOS or Android operating systems) where you can click on a map and the drone will go to that position; the map can be downloaded in advance. The app also has a number of one-click commands including take off, hover, return and land. Two sliding bars can be used to control the height and orientation of the drone; while another two control the camera gimbal with one tilting the camera up and down while the other pans it left and right. It’s micro control feature allows more precise movement of the drone over small distances. The Auto-Follow mode keeps the drone following the smart phone. If the drone exceeds 0.6 mile limit or the signal is lost it will return to its place of origin.

The app communicates with the drone via a G-box system, with the app communicating with the G-box via bluetooth while the G-box communicates with the drone via a wireless radio.

Other developments at higher Indiegogo targets include integration with a waterproof smartwatch and controlling the drone by tilting the smartphone, while obstacle avoidance will be available in future versions.

• Up to 20 mins flight time with the gimbal
• 10 mph (restricted) flight speed
• Up to 0.6 miles control distance
• Wind resistance up to 21 knot (11m/s)
• iOS and Android apps
• Propeller protectors

The Ghostdrone comes in 3 seperate versions, only two of which have a camera gimbal:

• Ghost Basic (£430) – No camera gimbal.
• Ghost Aerial (£550) – 2-axis gimbal designed for GoPro and similar cameras.
• Ghost Aerial Plus (£559) – 2-axis gimbal and Ehang Sports camera.

An RC controller can be purchased for an extra $99. Another optional extra are prop guards to protect both the system and what is may be flying near, and photos on the site seem to suggest that the mini legs attached to them can be used to replace the main landing gear, this would allow a greater field of view.

Potential
The Ghost Drone is marketed as the easiest done to fly, and the flight controls within the app will certainly enable many autonmated movements with one button click.

The propellors of the system are pointing down rather than up (unlike other systems), this is believed to provide more stable flight.

The fact that the Ghost Drone is wind and rain resistant means that is can be used in conditions that would ground other systems.

As with other systems, the follow-me technology will allow the easy recording of video site tours.

The Ghost Drone comes with an SDK (Software Development Kit) which has already been used by one of the users to create voice interface which can be used to control the drone.

Limitations
The Ghost Drone is limited to the abilities of smaller cameras.

The system does not come with an RC controller, although it is an optional extra for $99.

Although you can purchses prop guards the manufacturer suggests removing tham during filming as they destabilise the system in flight.

Samsung Gear VR

The Samsung Gear VR is a Virtual Reality headset designed to work with Galaxy Note 4 smartphone using Oculus Rift technology.

The headset uses an accelorator, gyrometer, magnetic and proximity sensors to enable interaction with a virtual environment by moving the headset using the same technology as the Oculus Rift.

On the side of the headset are a number of controls including a touch pad, back button, and a volume rocker. Focal adjustment can be also be undertaken on the system.

VR Content can be viewed through an Oculus Home App.

The systems has a 96º degree viewing angle.

The Innovator Edition is available in two editions; one which just consists of the Gear VR costs £185 with a second containing a Bluetooth gamepad for controlling content within the VR environment costs £240.

Potenital
Like other VR systems it could be used to remotely view immersive photos/videos of excavations/cultural heritage.

Unlike the Oculus Rift the system is wireless.

Limitations
The Gear VR is designed to only work with the Galaxy Note 4, if the user already owns one then it is only an additional of £185, but the phone itself costs £600 making it an expensive purchase for use with the Gear VR. Although technically it has greater potential than Google Cardboard the fact that it only works with one phone severely limits its user-ability.

The system cannot be connected to a PC so all material has to be downloaded via the phone.

DJI Inspire 1

The DJI Inspire 1 is the new quadcopter by the DJI company, this company is at the forefront of commercial innovations in UAV technology and the Inspire 1 brings a number of new technologies to the market.

One of these innovative design elements is that the rotors and legs can be lifted during flight allowing a full, unrestricted 360⁰ below the camera, this unrestricted view is common in much more expensive systems.

An option that is used in professional videography is dual-operator control, this is where one operator flies the UAV while the other controls the camera via the gimbal controls, with each operator having their own screen. The Inspire 1 has this option with the ability to buy two remote controls with the system as well as the technology within the drone.

Another innovative technology is the Optical Flow Technology which allows the location and altitude of the Inspire 1 to be maintained accurately even when a GPS signal is not available, such as an indoor environment, and allows the legs to automatically be lowered when landing. While the Vision Positioning System allows the system to hold its position while indoors, stop when the controls are released, and respond to your commands using a camera and ultrasonics system.

The DJI mobile app is available on iOS and Android and allows manual control of the camera, provides flight telemetry data and auto takeoff and landing controls.

The DJI Inspire 1 specifications are:

• Hovering accuracy with the GPS mode of 0.5m vertical and 2.5m horizontal
• Maximum Angular Velocity – Pitch: 300°/s and Yaw: 150°/s
• Maximun Tilt Angle: 35°
• Maximum Ascent Speed: 5m/s
• Maximum Descent Speed: 4m/s
• Maximum Speed: 22m/s (ATTI mode, no wind)
• Maximum Flight Altitude: 4500m
• Maximum Wind Speed Resistance: 10m/s
• Maximum Flight Time: Approximately 18 minutes

The camera specifictions are:

• 12 megapixel photographs
• Video in 4K at 24-30 fps, or 1080p at 24-60fps
• Lens with 9 elements in 9 groups including an aspherical element
• 1/2.3 inch CMOS sensor
• 94⁰ wide-angle FOV (Field of view)
• 3-axis, 360⁰ rotating gimbal

The video from the camera can be displayed in real-time in 720p HD on mobile devices thanks to DJI’s Lightbridge technology which can transmit video up to 2km.

The modular payload systen of the Aspire 1 suggested that there are planned options for other recording media.

The camera can be detached and attached to a pole making the camera a handheld recording device.

A SDK (Software DEvelopment Kit) has allowed outside company integration with both the camera and software.

The Inspire 1 costs £2,380 with one remote control and £2,749 with two remote controls.

Potential
The Inspire 1 is a complete ready to fly system.

A constant problem with cheaper UAV systems is the fixed nature of the camera, with gimbals only being able to pan up and down generally because of the position of the legs of the vehicle. More expensive systems have legs that can be lifted upwards by motors allowing the camera to move 360° from side to side meaning that a separate camera operator can control the camera while the pilot concentrates on the flying, but these tend to be very expensive (£5000-7000 or more) and still require the additional purchase of a camera. The Inspire 1 gets around this problem by designing a system which lifts the legs up at the same time as the props giving this 360° degree field of view.

The Optical Flow Technology allows the Inspire 1 to fly indoors with stability when there is no GPS signal to stabilize the system.

Thanks to these abilities the Inspire 1 has the ability to both map archaeology and cultural heritage by taking vertical overlapping photographs from above in a mapping pattern, to record upstanding monuments with the camera at a right angle to them, and to record flythroughs of both archaeology and cultural heritage with a separate controller panning the camera when required. Unlike other systems it provides all of these abilities in one package.

The ability to carry other recording packages in the future is very enticing, hopefully including lidar and infra-red.

Limitations
On the face of it the cost would seem too high, but after analysing what you get (see the article on RC Geeks), for full details it becomes quite reasonable.

• It comes with the DJI Lightbridge technology which would cost an additional £999 is purchased separately.
• To equip a previous UAV with the 4K camera technology would require the purchase of a GoPro Hero4 Black which costs £369.99.
• The Zenmuse H3-3D Gimbal to hold the camera would cost an additional £249.

So even buying the DJI Phantom 2 for £499 the additional costs to provide similar abilities to the Inspire 1 bring it close to the cost of that system, and that is without many of the abilities that the Inspire 1 has.

Although the Inspire 1 is pricey it comes with everything that you could need for professional style photography.

With the incareased movement of the many elements of the Inspire 1 there may be problems with wear and tear on the system.

Kodak PixPro SP360 Action Cam

The Kodak PixPro SP360 is an action camera aimed at the extreme sports recording market.

It is able to record 360º 10MP photographs and Full HD 1080p Video using its curved lens. Using the Pixpro Remote Viewer App it connects to other devices with iOS, Android or Windows operating systems using Wi-fi and NFC (Near Field Communication) allowing both control of the camera and viewing of the images/videos. The images/vidoes can be viewed in a number of different modes – ring, dome, front 180º and rear 180º, 360º panorama and globe. The system also has motion detector sensor which starts recording when motion is detected.

Much like the GoPro camera the system comes with a number of accessories for attaching it to different devices as well as a waterproof case which aare available in the different bundles.

The PixPro SP360 costs $349 for the Explorer bundle for entry-level and the Aqua bundle for watersports and $399 for the Extreme Accessories bundle.

Potential
The camera is able to record much more than a standard camera without the requirement for multiple cameras and the images/videos created can be viewed/exported in a number of different modes. It is water resistant, can be attached to a UAV or a person allowing a wide range of recording possibilities.

Limitations
Although the system can record a 360º view images/videos with the camera pointing upwards it is only able to record a 214º images/videos while pointing towards a subject. The distortion of a domed lens may also impact on the quality of the results.

HEXO+

The HEXO+ was a Kickstarter Project aimed at making aerial filming possible in many different areas without the need for another person controlling the UAV (Unmanned Aerial Vehicle) and camera. It is a hexocopter with six rotors designed to carry the GoPro camera on either a 2-axis gimbal or a 3-axis gimbal. It automatically flies itself and films the person holding the smartphone/tablet device which controls the UAV keeping them in frame as it flies.
http://hexoplus.com/

There is also an optional mount for another kickstarter project, the 360cam, which provides aerial 360° photos and videos.

The HEXO+ is controlled by an App on smartphones available in both iOS and Android versions. It can be set to film the person holding the smartphone from the front, side, back and anywhere in between, with the distance from the subject and altitude also being set. By using The Director’s Toolkit different filming scenarios can be configured, such as crane; pan, tilt, crab, dolly, 360° around you, far-to-close/close-to-far. Once configured the system can auto takeoff and land and will follow the subject maintaining the framing that was defined in the software. The system uses the Pixhawk autopilot system. It has a flight time of 15 minutes.

• Speed range: up to 70 kmh – 45mph
• Flight time of 15 min with gimbal attached
• Can fly in wind up to 15 mph
• iOS and Android apps

The system costs $949.00 with a 2-axis gimbal and $1,149.00 with a 3-axis gimbal.

Potential
Both the AirDog and similar system, the Hexo+, have the ability to follow a person carrying a smartphone or other device, keeping them in frame for the whole time frame of a video would seen to have great potential for the recording of site tours, which could now be recorded automatically from different altitudes showing the whole or parts of an excavation. The audio could be recorded with a digital recording device attached to the tour guide, with the audio and video being combined in post production.

They would also have the potential to record fieldwalking and exploration looking for new sites in remote regions.

The BBC has already begun to used UAV systems in the recording of news items – http://www.bbc.co.uk/news/business-24712136 – http://www.bbc.co.uk/blogs/researchanddevelopment/2012/04/collab-soton-uav.shtml – http://www.bbc.co.uk/programmes/n3csw972

Limitations
The HEXO+ is specifically designed for autonomous flight so it does not come with an RC (Radio Control) Controller, itthey can however both be switched to manual and an RC Controller bought separately can be used to control it like a standard UAV, although this obviously adds to the cost.

The recently released IRIS+ quadcopter has limited the usefulness of any UAV with “follow-me” technology, as not only is it a system with an RC controller and an autopilot that can be used to photographically map areas, but the system also has “follow-me” technology which matches that of other systems. It is also cheaper than the IRIS+.

The limitations of the system would be closely linked to the limitations of the GoPro camera which it uses to record.

The AirDog project rejected the use of smartphones for a number of reasons:

• Problems with using smartphones in extreme conditions – this is unlikely to be a problem.
• The average smartphone has only a 5-10m GPS accuracy horizontally, which is worse horizontally.
• Smartphones generally only have a 30-50m range for Wi-fi and Bluetooth which could cause potential problems if the UAV lost its signal – this would be less of a problem with site tour recording.

The six propellers of the HEXO+ make it the more stable of the two systems and more capable of landing if one motor were to fail although the extra motors will reduce the flight time.

360cam

The 360cam kickstarter project is a 360˚ Full HD camera, it is designed to take 360˚ horizontally and 300˚ vertically photographs and video with the systems 3 185˚ fisheye lens 8mpx cameras.
It provides images at a 4096 x 2048 resolution and video at 2046 x 1024 resolution at 30 fps h.264 mp4 and can record video for 60 minutes. The system can stitch images together in real time inside the camera, the files can also be exported to other 360˚ photo and video software. It also has the ability to stream Live video over WiFi.

During the recording process multiple images can be taken at different exposure levels providing high dynamic range photographs, which allows more variation in texture and colour to be recorded which can be used in the study of archaeology and cultural heritage.

The WiFi allows the 360cam to be controlled remotely with Android and iOS mobile phones and tablets.

http://www.360.tv/

One of the accessories for the 360cam is an underwater lens cup allowing 360˚ photographs and video to be taken underwater.

It can be easily attached to another kickstarter project, the HERO+ intelligent drone, with an optional mount for an additional $80.

With the 360cam Oculus Rift video player it is possible to view the immersive videos created using the Oculus Rift Virtual Reality headset.

The system costs $499.

Potential
The 360cam has great potential for the immersive recording of everything from archaeological excavations to walk-throughs of historic monuments. Its ability to be simply played in an Oculus Rift player adds significantly to the presentation possibilities of the video, although the current Oculus Rift Development Kit is another $350. The underwater lens cap allows the recording of photographs and video underwater adding potential for maritime archaeology.

Whether the overlap of the individual images can be used to create a photogrammetry model of the subject matter of the recording will have to be tested.

Limitations
The cameras of the 360cam are only 8mpx so the images provided would be of a much lower size than the similar Panono Camera.

Only a mount for the HERO+ drone can be purchased, but a mount for other UAVs could be printed using a 3D printer or manufactured in another way.

Google Cardboard

Google Cardboard are VR (Virtual Reality) goggles made out of cardboard using an android smart phone as the central processing unit and display via the Google Cardboard App. The cardboard shell can either be purchased for less than £10 complete with lenses and magnet button control or the cardboard design downloaded from the website and the other parts purchased separately.

The mobile phone provides you with orientation tracking using a gyroscope and accelerometer built into the phone which means that the the app can track the movement of the user’s head updating the imagery on the phone depending on the direction that the user is facing.

A magnetic trigger on the outside of the cardboard allows interaction with the VR environment by effecting the magnetometer in the phone. Although with calibrated magnetometers this can only act as a single button, with uncalibrated magnetometers incorporated in newer phone there is greater variety of abilities with the possibility of incorporating a joypad into the outside of the case.

The Google Camera App (On Android 4.4 and later) can record 360° Photo Spheres which can easily be viewed in the Google Cardboard App on the smartphone, other Photo Spheres can be viewed be editing their file names. The app can also view videos on YouTube including those designed for the Oculus Rift or other VR Gear with two separate views in the video. Integration with Google Earth and Google Maps Street View is also possible.

Two SDKs (Software Development Kits) can be downloaded from the website:

• The first is the Cardboard SDK for Android which allows VR applications to be quickly created in OpenGL.
• The second is the Cardboard SDK for Unity which allows an application created in the Unity 3D game engine to be viewed in the Google Cardboard Goggles or to design one from scratch.

Although designed for phones with the Android operating system, phones using iOS can also be used in the Google Cardboard using Durvois Dive (a plastic VR Goggle frame which also uses smartphones) apps.

Potential
Google Cardboard was designed to both allow the cheap and easy ability for almost anyone to view VR and to help push forward development of the systems.

It has the ability to both view virtual reconstructions of sites and view still 360° photographs and immersive videos of sites, these can easily be downloaded and viewed by anyone anywhere in the world using the technology. The fact that the phone can be used to create 360° Photo Spheres as well enables the both the recording and viewing of views of cultural heritage and excavations with technology that may already be owned.

Limitations
Because it uses a smartphone there are limitations to its abilities that would not be there with more powerful computer systems. The quality of the imagery is also completely dependent on the quality of the smartphone screen.

Although the control is limited to the one button on the outside of the Google Cardboard some wi-fi/bluetooth game controllers can be used with Android operating systems allowing much more interaction. There have however been problems with the button working, particularly on certain models, it is after-all a technical workaround to use a device for a function it was not designed for.

The Pocket Drone

The Pocket Drone is a Kickstarter Project collapsible tricopter.

The system can fold up smaller than a 7in tablet.

Its specifications are:

• Unique collapsible compact design
• APM autopilot
• Integrated onboard autopilot Flight planning with Google Maps
• Compatible with Mission Planner or QGroundControl software
• Fly by GPS waypoints, altitude hold and return to home functions
• “Follow me” mode
• 20 mins flight time with onboard camera
• Innovative swing arm and telescoping tail frame
• Folding propellers
• Built-in landing gear
• Built-in camera mount
• Travel case included
• It can be controller the included controller or a DSM (Digital Spectrum Modulation) compatible radio controller
• Control from any Android tablet or phone with a USB port and GPS, iOS compatibility coming soon
• PC, Mac and Linux compatible
• Weighs approximately 1 pound
• Can carry an action camera like the GoPro with its outer case
• Can carry up to 1/2 pound payload
• Deploys in less than 20 seconds
• Quick recharge system

The Kickstarter backer version of the drone cost between $415 and $596 depending on whether the user wanted a controller and action camera.

Potential
Its collapsible nature means that it can be easily transported and the fact that is can be deployed in less than 20 seconds means that recording can be begun very quickly.

Limitations
It is limited to the abilities of action cameras.

The camera does not appear to be stabilized so video footage may be shaky.

Panono Camera

The Panono is an Indiegogo crowdsourcing project to create a camera ball which records 108 megapixel, 360° x 360° full-spherical panoramic images with 36 separate cameras built into the ball. It is thrown into the air and an accelerometer calculates when it is at the peak of its height and takes the photographs. Photographs can also be taken with the Panono in hand or mounted on a pole.

The results can be previewed in the iOS and Android Panono App with stitching available in the Cloud.

http://www.panono.com/ballcamera/

The Panono Camera costs €549.

Example
An example panono panorama of Grunewald Tower in Berlin taken with an octocopter (drag mouse to rotate image).

Potential
The Panono camera has potential to aid in recording, post-excavation work and and visualisation of sites. Its ability to quickly record 360° panoramic images can allow important/complicated areas of an excavation to be recorded in a moment and later viewed when post-excavation work is being carried out allowing the easy visualisation of what is being written up. They can also be put on websites to show the site to the public and aid with their interaction with archaeology.

Limitations
The camera is aimed at use by the public for recording things they find of interest and is of limited use as it only takes static photographs in one place unlike the similar 360cam.