Tag Archives: photograph

UAV Building Facade Recording – Part 1 – Preliminary Ideas and Experimentation

The recording of buildings is an important area in Cultural Heritage, whether for conditional surveys or to record something that is about to be destroyed.

Traditional methods rely upon survey equipment such as Total Stations to take a number of points on the façade, but this results in only points and lines with no great surface detail.

Other more detailed survey techniques such as laser scanning and photogrammetry have also been employed. But laser scanning is expensive and both the techniques are generally ground based missing detail of the façade that is not visible from this position. Scaffolding or a cherry picker can be used to record the whole of the building but again this can add to the cost the recording.

Photogrammetry is a low cost method of producing high quality results but relies upon having the camera parallel to the building to produce the best results, as capturing photographs from an angle brings inaccuracies into the recording as well as there being more detail at the bottom of the 3D model created than at the top.

The UAV would seem to provide an ideal platform to carry a camera parallel to the building, recording photographs with the required photogrammetry overlap. And with its autopilot it would seem possible to automate the recording process allowing the mapping of the façade in the same way that the UAV  can map the ground.

There are of course a number of problems that need to be overcome.

Building Façade Recording

Manual

Building façade recording can be done manually with a UAV, but the larger and more complicated the building façade the mode difficult it is to do this accurately. As the pilot needs to control the UAV accurately in 3 dimensions as well as controlling its speed.

Although the results for an experimental UAV mission are acceptable the difficulty of maintaining a manual position can be seen in the image below.

Automatic

In order to automate the process you need to determine what parameters are required to record a building façade using photogrammetry.

These can be seen below.

Excel Calculations

Building facade recording parameters

First experimentation was done by taking the co-ordinates of the two ends of an example the wall from Google Earth (The south facing wall of the lay brothers’ quarters at Waverley Abbey in Surrey was used). These co-ordinates can then be used to determine the bearing that the wall lies upon and its width. Using the camera parameters and level of detail the required distance from the wall for the flight can be calculated using trigonometry. Trigonometry is once again used to calculate the offset positions for the left and right extent of the flight.

 

The image overlap can be used to determine the number of photographs required in the horizontal and vertical, and hence the change of altitude that is required for each flight pass of the building.

Calculate altitude changes

Calculate altitude

Although it is planned to have the ability for the UAV to hover and take photographs, it is much easier to have it take photographs as it flies across the building façade. This requires the additional calculations and control of optimum flight speed and shutter speed to take photographs which are not adversely effected by motion blur.

Shutter speed formula

Shutter speed formula

Shutter speed calculions

Shutter speed calculations

These preliminary calculations were done in Microsoft Excel.

DroneKit

The drone manufacturer 3DR provides a series of software development kits (SDKs) for writing applications to control your UAV using one of the open-source autopilot systems they support.

DroneKit Python uses the Python programming language and provides a number of examples to help with programming the flight of a UAV; these include flying from co-ordinate to co-ordinate up to complete missions. Together with this there is an API (application program interface) reference which provides all of the Python commands that can be used to control the UAV.

Python

Python is a fairly easy to learn programming language and as DroneKit already requires it to be installed and setup it makes sense to use the same language to calcuate the required paramaters for the flight path. This was done with the aid of a number of online resources. A graphical user interface (GUI) was created using the Tkinter Python package and was used to enter the data. The python code did the calculations then a file is exported which combines these calculations with the DroneKit code for controlling the autopilot. The final file when run will control the UAV flight.

Python GUI

Python GUI

Virtual Drone

Experimentation doesn’t need to be done with a live UAV, it can actually be done with a virtual one using a number of pieces of open-source software. These include Mission  Planner, ArduCopterMAVProxy and SITL (Software in the loop)

Virtual Drone

Virtual Drone

Next Steps

Experimentation with a UAV using the hardware and software is the next step to test whether a GPS can be used in close proximity to a structure.

Limitations of standard UAV GPS accuracy to within the range of meters also complicates the use of this method of controlling the flight. This either needs to be solved with the use of a more accurate GPS (although the proximity to the building may block the signal), sensors that measure distances or the use of computer vision technologies to control the UAV position. The UAV afterall currently only need to fly between two set points then at set altitudes above the ground.

Advertisements

High Dynamic Range Photography/Photogrammetry – Part 1

High Dynamic Range (HDR) Photography

High Dynamic Range is a popular photographic technique that is used to produce more realistic photographic results or artistic images. It is a technique that can be used to try and replicate what the human eye can see as the dynamic range of a camera is limited and it is unable to record the lightest and darkest elements in a single photograph. This can be remedied by taking a number of photographs with varying shutter speed/aperture combinations and combining them using specialist software to produce a photograph with a greater dynamic range than can be recorded in a single photograph.

Many new digital cameras have the ability to produce HDR photographs using Auto Exposure Bracketing (AEB), special HDR settings (this may process the images for you resulting in an HDR photo on the camera but losing the originals) or manually setting the camera up.

Limitations of archaeological and cultural heritage photography

An intrinsic problem with taking photographs in archaeological and cultural heritage contexts is lighting; both too much and too little lighting are factors that hamper recording images that include as much detail as possible.

In the case of archaeological excavations attempts have been made to limit the problems in section photography by either reflecting more light in using white card or using a tarpaulin to cast a shadow. Both of these techniques work but require time and manpower.

The same problem is encountered in building recording where in an outside environment strong lighting can cause both bleached out areas and heavy shadows.

Netley Abbey - East Window

Netley Abbey, Hampshire – East Window. Photograph demonstrating the problem of strong oblique lighting causing both too much and too little light in the same image.

While lighting through windows can cause similar problems on the inside of buildings.

Netley Abbey, Hampshire - Sacristy/Library.

Netley Abbey, Hampshire – Sacristy/Library. Photograph demonstrating the effect of excessive light coming through a window causing the dual problems of both too much light near to the window and too little light in other areas.

In order to reveal elements in dark shadow a high exposure camera setting is required, while bright bleached out areas are only revealed with low exposures. These elements together with well lit areas cannot by revealed in one single photograph, this is where High Dynamic Range photography comes in.

HDR Photography in Archaeology

HDR Photography was introduced to Archaeology by David Wheatley in 2010, he provided examples of its use in improving the standard recording methods of excavations, cave sites and even using archived analogue archaeological photographs. Sadly its use was not embraced by the community probably due to technological limitations at the time and the inherent conservatism of the industry and museum archives which were yet to embrace digital photographic technology. Technology and the industry have now caught up with his ideas, with digital cameras being present on most if not all excavations, while other scholars have now begun to bring the technology to the technique of 3D recording using photogrammetry.

HDR Field Archaeology Photography

Photography is one of the primary recording techniques within field archaeology and has been since the introduction of discipline, but conservatism within field archaeology has meant that it was only fairly recently that digital photography became the primary recording technology.

Digital cameras have a number of benefits within archaeology:

  • The ability to take numerous photographs on one memory card.
  • No need to pay to process films.
  • No need to digitize the photographs.
  • Where once excavators may have been told to limit the number of photographs taken on an excavation to keep the processing costs down, digital media allows almost limitless photographs to be taken.
  • Photographs can be as easy as point and click with the camera controlling all of the settings.

But they also have drawbacks:

  • Where once archaeologists knew how to use an analogue camera to take bracketed shots, the automatic setting on digital cameras is commonly the only setting used as it produces results at a required level of quality, this means that the archaeologist may not know how to properly operate the camera.
  • Although almost limitless photographs can be taken, limits should be included as the archive may still need to be sorted through.
  • The requirements for digital storage can be complicated and costly.

Although not ideal, a number of modern cameras now come with an HDR setting on them which in many cases can be changed to the required level of bracketing, although only the merged photograph is saved losing the possibility of later re-processing the photographs with different settings.

Field Archaeology Archive Photographs

One benefit of traditional bracketing of analogue photographs for archaeological excavations is that they provide an ideal resource for conducting HDR processing. These archives have multiple photographs at different exposure levels which can be digitized and processed to provide better results than the originals and be re-entered into the archive with the digitized originals.

HDR using archive slides from excavations at the Cove, Avebury using archive slides (Wheatley 2010)

HDR using archive slides from excavations at the Cove, Avebury (Wheatley 2010)

HDR Building Photography

Building recording is an area that can be significantly enhanced by the use of HDR. It is difficult to provide adequate lighting in many cases, meaning that some areas are brightly illuminated while others are dimly lit loosing information in both cases.

Processing bracketed images into an HDR image provides a greatly enhanced image.

HDR Photogrammetry

Recent developments in camera technology, HDR software and photogrammetry software have allowed the introduction of HDR Photogrammetry. Thanks to the additional information present in the photographs models of higher detail and accuracy can be created in non-optimal lighting conditions.

As well as the ability to use tone mapped images produced from HDR images the Agisoft PhotoScan Photogrammetry software can also process .exr file format High Dynamic Range images into 3D models.

HDR Object Photogrammetry

One area under study is its use in photographing objects. The benefits are determined by the type of material used, some are greatly enhanced by HDR while others are little altered.

Image matching result from the images originated with different HDR processing: a) No HDR; b) tone mapped images from HDR processing (Guidi et al 2014)

Image matching result from the images originated
with different HDR processing: a) No HDR; b) tone mapped
images from HDR processing (Guidi et al 2014)

HDR Building Photogrammetry

We have already seen the benefits of HDR Photography in building recording and this can continue with photogrammetry.

Netley-Window-PG

Photogrammetry point cloud of the east window of Netley Abbey, Hampshire showing how the raking sunlight on the left-hand side of the window has bleached out the photographs and lost detail

Both the increased level of quality of the photograph and the higher amount of detail present in the 3D model can easily be seen in the HDR photogrammetry model.

Software Solutions

A number of software solutions are available for the processing of HDR photographs, these range from high end photographic software such as Adobe Photoshop and Lightroom, through to HDR specific pieces of software and even open source solutions. HDRSoft’s Photomatrix comes in a number of versions which include plugins for different software packages such as Adobe Lightroom, Photoshop Elements, Photoshop and Apple Aperture. With low cost solutions such as Fusion HDR or free open-source solutions such as LuminanceHDR also being available.

In order to be view-able on low contrast monitors and paper the images need to go through a process called tone mapping, this replicates the appearance of the high dynamic range photograph on these media.

Downloaded imaged can be batch processed in software such as Photomatrix setting up how many images need to be merged together with a number of preset or custom settings allowing the images to be processed exactly as required. These pieces of software can also compensate for slight movement between the recording of the multiple images. The resulting images can then be saved as either .hdr (Radiance) or .exr (OpenEXR) file formats which record the HDR information.

Batch processing of images within Photomatrix

Batch processing of images within Photomatrix

Benefits
HDR photography can record more information in both photographs and photogrammetry models. By using open Source HDR software it can be free. Many cameras allow multiple bracketed photographs to be be taken automatically only adding a few seconds to the recording process.

It is also possible in some of the software to open a folder full of images and have the software batch process it without any user intervention once the preset settings have been loaded.

Drawbacks
Among the drawbacks are the fact that as the camera is taking multiple photographs it is difficult to stabilize the camera by hand, otherwise there will be movement between the photographs. Although movement between photographs can be corrected if you are bracketing shots and using software the automatic HDR setting on the camera will probably result in a blurry image.

UAV HDR Photogrammetry

UAV HDR Photogrammetry is an area I will be studying in the future. It has great potential for recording but will require a careful balance of UAV hovering, a steady gimbal, fast shutter speed and an adequate depth of field. It will be discussed in a future blog.

Sources
Guidi, G., S. Gonizzi, and L. L. Micoli. “Image pre-processing for optimizing automated photogrammetry performances.” ISPRS Annals of The Photogrammetry, Remote Sensing and Spatial Information Sciences 2.5 (2014): 145-152.

Kontogianni, G., and A. Georgopoulos. “Investigating the effect of HDR images for the 3D documentation of cultural heritage.” World Cultural Heritage Conference 2014 – Euromed 2014 – International Conference on Cultural Heritage Documentation, Preservation and Protection. (2014)

Ntregkaa, A., A. Georgopoulosa, and M. Santana Quinterob. “Photogrammetric Exploitation of HDR Images for Cultural Heritage Documentation.” ISPRS Annals of the Photogrammetry, Remote Sensing and Spatial Information Sciences 5 (2013): W1.

Wheatley, D. “High dynamic range imaging for archaeological recording.” Journal of Archaeological Method and Theory 18, no. 3 (2011): 256-271.

Cloud UAV Photogrammetry Processing

Online photogrammetry processing systems such as Autodesk’s 123D Catch (which now forms part of a whole suite of software designed to do everything from 3D capturing to 3D model creation through to 3D printing) have been around for a number of years and are available on platforms including iOS, Android and Windows. But they are aimed at the amateur with limited settings and low quality results.

Recent developments in Cloud Photogrammetry Processing have brought developing technologies that can potentially save a lot of time and money in processing photographs taken on site and in the office having the abilities of commercial desktop software solutions as well as producing high quality results.

Commercial Solutions

The DroneMapper company are one of a number of ventures aimed at the processing of UAV (Unmanned Aerial Vehicle) photographs for a number of industries including archaeology. Rather than using existing photogrammetry solutions they have developed their own custom in-house photogrammetry software package.

Images or a RAR archive file can be uploaded to their server using either their web interface, FTP (File Transfer Protocol) interface or a Dropbox account. Their current processing costs are between $20 and $100 USD.

While the REDCatch company provides processing of ground based and object processing as well as UAV photographs ; costing anything from 290€ to over a 1000€

Open Source Solutions

An Open Source alternative to this is Open Drone Map, this system uses a number of previously developed SFM (Structure from motion) solutions to automate the processing of photographs into 3D models, orthophotos and Digital Elevation Models (DEM) for GIS (Geographic Information Systems) applications. Although free for non-commercial purposes a license needs to he purchased to use it in commercial circumstances.

OpenDroneMap running on Ubuntu Linux

OpenDroneMap running on Ubuntu Linux

The code can be downloaded from Github and includes detailed written instructions and YouTube videos on how to install if on Ubuntu Linux. This means that it can be simply installed on Ubuntu running on many internet hosting services.

It uses both Clustering Views for Multi-view Stereo (CMVS) and Patch-based Multi-view Stereo Software
(PMVS)
developed by Yasutaka Furukawa and Jean Ponce; as well as Bundler: Structure from Motion (SfM) for Unordered Image Collections developed by Noah Snavely.

hall-all

CMVS

teaser

PMVS

Colosseum

Bundler

The VisualSFM GUI developed by Changchang Wu includes the same software combined into a single computer program.

Potential
As we can see there is great potential for the cloud processing of photogrammetry models, whether by commercial companies or with open source software. This can remove the need for expensive photogrammetry software and the expertise to use it. Photographs can be uploaded as they are taken, or shortly afterwards and the processing begun before the person recording leaves the site. This obviously saves the time it would take to return to the office and download the photographs.

Limitations
One current limitation of the open source system is the fact that it is solely aimed at photogrammetric vertical mapping recording, meaning that there is no need to mask the photographs in the software. There is however a requirement for the masking of photographs within the photogrammetric recording of standing structures and objects where elements of the photographs need to be masked out in order to get the best results.

Bibliography
Agarwal, Sameer, Noah Snavely, Steven M. Seitz, and Richard Szeliski. “Bundle adjustment in the large.” In Computer Vision–ECCV 2010, pp. 29-42. Springer Berlin Heidelberg, 2010.

Agarwal, Sameer, Yasutaka Furukawa, Noah Snavely, Ian Simon, Brian Curless, Steven M. Seitz, and Richard Szeliski. “Building rome in a day.” Communications of the ACM 54, no. 10 (2011): 105-112.

Furukawa, Yasutaka, Brian Curless, Steven M. Seitz, and Richard Szeliski. “Towards internet-scale multi-view stereo.” In Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on, pp. 1434-1441. IEEE, 2010.

Furukawa, Yasutaka, and Jean Ponce. “Accurate, dense, and robust multiview stereopsis.” Pattern Analysis and Machine Intelligence, IEEE Transactions on 32, no. 8 (2010): 1362-1376.

Wu, Changchang, Sameer Agarwal, Brian Curless, and Steven M. Seitz. “Multicore bundle adjustment.” In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp. 3057-3064. IEEE, 2011.

Wu, Changchang. “Towards linear-time incremental structure from motion.” In 3D Vision-3DV 2013, 2013 International Conference on, pp. 127-134. IEEE, 2013.

PlexiDrone

The PlexiDrone is an Indiegogo drone project aimed at film makers and aerial photographers.

The system comes in two different configurations; the x4 has four rotors while the x8 has 8, four above and four below. As a result they have different flight characteristics.

  • Flight time – 25 mins x4, 20 mins x8
  • Speed – 37 mph x4, 42mph x8
  • Thrust – 2.35 kg x4, 4.1 kg x8

It’s other specifications include:

  • Follow me technology
  • Retractable landing gear allowing an unobstructed view below the drone enabling a 360˚ camera gimbal
  • SwarmTech allowing multiple drones to be controlled and film the same subject matter
  • Object avoidance using ultrasonic sensors which can sense up to 32 ft. (10 m)
  • The market version will include a bottom facing ultrasonic sensor which will help control altitude and lower the landing gear when required
  • It will also include a front-facing and bottom facing optical flow sensor helping with stability when indoors and without a GPS signal
  • The PlexiDrone allows modular payloads including:
    1. PlexiDrone 3D Gimbal which carries a GoPro
      PlexiCinema 3D Gimbal which carries mirror-less cinema-quality cameras like the Sony A5000, Sony RX100 III, and Lumix GM5
      Bubl 360˚ Camera
      Sony ActionCam
  • PlexiGCS control App on iOS, Android or Windows
  • Snap together construction fits into the PlexiPack or PlexiPack Mini custom made backpack
  • PlexiFPV Real-time video which simply snaps onto the PlexiDrone
  • The system will work with standard RC (Radio Control) controllers
  • 3-axis camera stabilisation

The PlexiDrone comes in a number of buying packages as a quadcopter and as an octocopter.

  • The Standard package comes without a gimbal for $699
  • The Pro package comes with a gimbal for $799
  • The Swarm Pro pack with 3 PlexiDrones and a PlexiHub to control them for $2,499
  • The 3D gimbal can be purchased for $159 which will carry a GoPro or other action camera
  • The PlexiCinema 3D gimbal can be purchased for $349
  • The PlexiFPV first person view system can be purchased for $215
  • The system can be upgraded to an octocopter for $199 by replacing the single rotor snap-on arms with dual ones

Potential
Due to the fact that the PlexiDrone is aimed at aerial photography rather than another discipline it is better suited to the recording of Cultural Heritage and Archaeology. As well as having a number of different photographic platforms available it also has technology such as follow me, obstacle avoidance, optical flow sensors and swarm capabilities which add to its potential for recording in different environments. Some of these technologies are only available on the much more expensive DJI Aspire 1.

The retractable landing gear allows for much more freedom of movement in the camera gimbal which can record 360˚ around the horizontal access and 180˚ in the vertical without any of the drone appearing in the photographs.

The snap together construction of the PlexiDrone allows for easy packing into a backpack style storage system which appears easily portable.

The fact that the system has easily interchangeable payloads allows for much more flexibility in recording than other systems. It can carry a GoPro like all of the other cheap drone systems, but can also carry heavier cameras with better optics and higher megapixels allowing greater quality recording and it can also carry the bubl 360˚ camera as well adding 360˚ recording to its abilities.

The swarm capabilities allows multiple drones to record the same subject from different directions, increasing the speed of recording and the amount recorded.

Limitations
Using the swarm capability obviously adds significantly the cost.

Although it can carry more complex digital cameras it cannot carry the heavier high-spec digital SLR cameras.

Ghost Drone

The Ghost Drone is a wind and rain resistant Indiegogo project drone aimed at filmmakers, photographers, sports enthusiasts, travelers and adventurers, GoPro owners and first time and experienced drone pilots.

It is controlled by a smartphone app (either iOS or Android operating systems) where you can click on a map and the drone will go to that position; the map can be downloaded in advance. The app also has a number of one-click commands including take off, hover, return and land. Two sliding bars can be used to control the height and orientation of the drone; while another two control the camera gimbal with one tilting the camera up and down while the other pans it left and right. It’s micro control feature allows more precise movement of the drone over small distances. The Auto-Follow mode keeps the drone following the smart phone. If the drone exceeds 0.6 mile limit or the signal is lost it will return to its place of origin.

The app communicates with the drone via a G-box system, with the app communicating with the G-box via bluetooth while the G-box communicates with the drone via a wireless radio.

Other developments at higher Indiegogo targets include integration with a waterproof smartwatch and controlling the drone by tilting the smartphone, while obstacle avoidance will be available in future versions.

  • Up to 20 mins flight time with the gimbal
  • 10 mph (restricted) flight speed
  • Up to 0.6 miles control distance
  • Wind resistance up to 21 knot (11m/s)
  • iOS and Android apps
  • Propeller protectors

The Ghostdrone comes in 3 seperate versions, only two of which have a camera gimbal:

  • Ghost Basic (£430) – No camera gimbal.
  • Ghost Aerial (£550) – 2-axis gimbal designed for GoPro and similar cameras.
  • Ghost Aerial Plus (£559) – 2-axis gimbal and Ehang Sports camera.

An RC controller can be purchased for an extra $99. Another optional extra are prop guards to protect both the system and what is may be flying near, and photos on the site seem to suggest that the mini legs attached to them can be used to replace the main landing gear, this would allow a greater field of view.

Potential
The Ghost Drone is marketed as the easiest done to fly, and the flight controls within the app will certainly enable many autonmated movements with one button click.

The propellors of the system are pointing down rather than up (unlike other systems), this is believed to provide more stable flight.

The fact that the Ghost Drone is wind and rain resistant means that is can be used in conditions that would ground other systems.

As with other systems, the follow-me technology will allow the easy recording of video site tours.

The Ghost Drone comes with an SDK (Software Development Kit) which has already been used by one of the users to create voice interface which can be used to control the drone.

Limitations
The Ghost Drone is limited to the abilities of smaller cameras.

The system does not come with an RC controller, although it is an optional extra for $99.

Although you can purchses prop guards the manufacturer suggests removing tham during filming as they destabilise the system in flight.

Zano

The Zano is a Kickstarter project Nano drone.

Amongst its features are:

  • 5 megapixel HD video camera
  • IR obstacle avoidance
  • Echo sounding sonar and high resolution air pressure sensor for altitude control
  • Follow me
  • Bidirectional motor control (Zano can drive motors in either direction)
  • iOS and Android compatible app
  • 10 – 15 minute flight time
  • 15 – 30 meter optimal operating range
  • 25 mph top speed
  • Can fly in “Free Flight” mode, using on-screen Joy Sticks). On screen slide bars control rotation and altitude
  • ZANO will hold its position unless instructed otherwise
  • Tilt control of the Zano using a smart device
  • Automatic return to smart device

Other project developments include:

  • Tracking of a set target through image processing
  • Facial recognition capability
  • 360 and 180 degree Panoramics
  • Swarming capability

The app will also allow the purchase of future developments of the system.

The ZANO will cost £169.95.

Potential
It’s different technologies allow number of different applications including site tours and recording videos of sites. The swarming capability opens up the possibility of recording sites with a number of different drones at the same time. While it’s size makes it very portable and easy to use in confined spaces allowing the recording of areas that other drones may not be able to reach.

It comes with a with two part SDK (Software Development Kit) which allows development of apps using the core functionality of the drone and even control of the system using a VR headset.

Limitations
It’s camera is quite low quality so the photographic/video results may not be high quality.

Samsung Gear VR

The Samsung Gear VR is a Virtual Reality headset designed to work with Galaxy Note 4 smartphone using Oculus Rift technology.

The headset uses an accelorator, gyrometer, magnetic and proximity sensors to enable interaction with a virtual environment by moving the headset using the same technology as the Oculus Rift.

On the side of the headset are a number of controls including a touch pad, back button, and a volume rocker. Focal adjustment can be also be undertaken on the system.

VR Content can be viewed through an Oculus Home App.

The systems has a 96º degree viewing angle.

The Innovator Edition is available in two editions; one which just consists of the Gear VR costs £185 with a second containing a Bluetooth gamepad for controlling content within the VR environment costs £240.

Potenital
Like other VR systems it could be used to remotely view immersive photos/videos of excavations/cultural heritage.

Unlike the Oculus Rift the system is wireless.

Limitations
The Gear VR is designed to only work with the Galaxy Note 4, if the user already owns one then it is only an additional of £185, but the phone itself costs £600 making it an expensive purchase for use with the Gear VR. Although technically it has greater potential than Google Cardboard the fact that it only works with one phone severely limits its user-ability.

The system cannot be connected to a PC so all material has to be downloaded via the phone.