Volume measurement in two-phase flow systems using computer vision:
a condensed overview

This work provides a concise summary of my research paper submitted to the Students’ Scientific Conference (TDK) organized in 2017 by the Faculty of Electrical Engineering and Informatics (VIK) at the Budapest University of Technology and Economics (BME). The paper was awarded second place within the Intelligent Systems section. The original version can be accessed in Hungarian on the official TDK website.

Introduction

Automation has a significant impact on society and the economy. It is desired in many areas for addressing the limitations of manual labor, increasing productivity and providing consistent quality and output. Moreover, the advancement of technology further drives its incorporation through reducing the costs of adaptation and enabling the automation of an expanding range of tasks. The latter is greatly exemplified by computer vision.

Computer vision

Computer vision is a rapidly growing field devoted to analyzing, transforming and high level understanding of images [1]. It is used in a diverse set of domains: cameras can detect and focus on faces; video call applications can blur the background to increase user privacy; programs can transcribe text from pictures; soldering errors and missing parts can be detected on printed circuit boards [2], [3]; faulty products can be identified in the food industry by inspecting the shape, volume, surface and the color [4], [5].

Alongside its widespread usage, it is noteworthy that the adaptation of computer vision happens at different rates across various fields. This roots primarily in the interdisciplinary nature of this task, requiring specialists and cross-industry collaboration. Flow chemistry is a good example where, despite the high level of integration of information technology, computer vision based applications are rare.

Flow chemistry and Taylor flow

Flow chemistry has emerged in the last decade as an enabler technology that can provide an enhanced safety profile for reactions that involve hazardous or explosive intermediates, and for processes which involve high temperatures or pressures [6]. Furthermore, it often facilitates rapid reactions with short-lived reactive intermediates [6]. Compared to batch processes, flow systems allow finer control over the reaction parameters and conditions. With proper monitoring tools, systems can be automated to scan a wide range of parameters in order to find the optimum [7].

In the past decades, the study and industrial application of gas-liquid two-phase flow in small-channel systems have become a significant area [8], [9], [10], [11], [12]. One of the dominant flow regimes (flow patterns) in such systems is the Taylor flow [13], [14], [15], which consists of elongated bubbles separated by liquid slugs (liquid plugs), as illustrated in Figure 1. The hydrodynamic properties of the Taylor flow make it attractive for carrying out gas-liquid operations [13], resulting in its usage within various industrial processes [14]. For understanding the behavior and improving the performance of chemical systems that operate in this regime, knowing the flow characteristics is crucial [13]. This requires the measurement of the flow parameters [12], such as - but not limited to - phase velocity, bubble and slug lengths, and the void fraction [12], [13], [14]. However, the small-channel flow characteristics complicate the measurement of these compared to normal-scale channels, thus knowing which techniques to apply is critical [12]. The potential for automating the measurement of flow parameters of small-channel gas-liquid two-phase flow systems is illustrated through the example of a recently published chemical application [6].

A British research team has developed a reactor, based on a semi-permeable Teflon® AF-2400 membrane, which allows the advantageous execution of catalytic hydrogenation reactions at elevated pressure in flow within organic chemical synthesis. The device enables both the uptake of hydrogen into the solution and the removal of it. The quantitative measurement of the gas permeation used the traditional burette method at first, but was later simplified with a novel computer vision based “bubble counting” approach. The setup is depicted in Figure 2.

Hydrogen has a low solubility in the used dichloromethane (DCM) solvent at atmospheric pressure. Therefore the amount of gas venting from the solution downstream of the back-pressure regulator, when the total pressure could relax to atmospheric pressure, is a reasonably accurate measure of the gas absorbed by the flow stream in the reactor. By adding red dye to the solvent, the gas and the liquid could be distinguished by color. A section of tubing wrapped several times around a frame was held in place in the view of a camera. Using a calibration photo and basic image processing, approximating the concentration of hydrogen was reduced to counting the number of red and white pixels, as seen in Figure 3.

Since the camera, the pump system and the pressure regulator can be placed under computer control, the measurement can be fully automated. Such bubble counting application can facilitate the measurement of gas inlet system efficiency and equilibrium gas solubility data on a wide range of substances, while being simpler and faster to carry out than many currently available methods [17], [18], [19], [20].

However, the approach has certain limitations: it relies on a colorful dye, requires a long tube section (1.54 m), stops and waits for the flow to relax before taking the pictures and can only directly measure the concentration of hydrogen. My goal was to create an application that can overcome these limitations and effectively measure the volume of passing gas in small-channel gas-liquid two-phase flow systems operating in the Taylor flow regime.

Developing the application

The development of the application was an iterative process, during which the experimental setup and the program evolved side by side. The setup enabled the recording of videos and reference values, supporting the evaluation of a diverse set of approaches and algorithms; this evaluation, in turn, guided the refinement of the setup. The overview of the final experimental setup, processing algorithm and implementation are presented in the following sections.

Experimental setup

In the final setup, air and deionised water were pumped separately through two peristaltic pumps (1.2 ml/min individual flow rate) into a common tube. This tube passed through the video recording section to a burette. (The pumps and burette were regular lab equipment, their operation described in detail within my paper.) The videos were captured in a resolution of 1280x720 pixels and 50 frames per second (fps) using a Canon EOS 70D camera equipped with Canon EF-S 60mm f/2.8 USM lens. The configuration is illustrated in Figure 4.

The video recording section had an effective length of 7 cm and was constructed from a backlight source, a plastic ruler, a heat-resistant circular glass tube and pieces of black paper. The backlight was crucial: it provided consistent lighting conditions that eliminated shadows and other environmental noises, while also enhancing the visibility of the examined tube section. The plastic ruler acted as the reference, enabling the conversion of length between pixels and millimeters. The heat-resistant circular glass tube provided a fully transparent and mechanically stable tube section. The latter was important as, at low flow rates, the pulsating nature of the peristaltic pumps would make a flexible pipe move. The internal diameters of the tested tubes were 2.8 mm, 3.95 mm, 4.8 mm, and 8 mm, respectively. The channel width of 8 mm was too big for the Taylor flow to form. Regarding the others, two observations were made. Firstly, the bigger the diameter is, the more visible the phase boundaries could be made. Secondly, the smaller the diameter is, the more rotationally symmetrical the bubbles become. (The ends of a bubble can bend upward in a horizontal tube.) The tube with the diameter of 3.95 mm provided a good compromise and was chosen for the final setup. Lastly, pieces of black paper were laid parallel to the tube for enhancing the contrast of the phase boundaries. While analyzing some recordings, it was observed that as the bubbles are passing along an object near the tube, its reflection appears in the border of the bubble. This turned out to be a cornerstone of the application: the bubble-edges could be darkened and made easily visible. See Figure 5 for a snapshot of a recorded video.

Algorithms

The processing algorithm tasked with calculating the volume of the gas consisted of three main sections: the location, the tracking and the measurement of the gas bubbles.

The bubble location built on the gas-boundary darkening technique of the experimental setup. Using a reference image taken of the water-filled recording section, the bubble outlines could be located with grayscale image differencing. To ensure having one contiguous contour per bubble, the fragmented outlines could be united with image processing techniques such as thresholding and morphology; this is demonstrated in Figure 6.

Using the spatial overlap of the areas outlined by the contours across consecutive video frames, the bubbles could be identified and tracked. The contours on the new frame with no suitable overlaps with already tracked objects belong to newly sighted objects, while tracked objects without suitable overlaps on the new frame do disappear. This overlap-based approach would also be capable of handling the bubbles splitting up and merging together, had it been necessary.

The gas volume measurement relied on the assumption that the elongated gas bubbles could be approximated as cylinders with half-spheres at their ends. (This is backed up by [14] for low capillary numbers, although it was noticed only when writing this summary.) The volume of a bubble, therefore, would be equal to the volume of a cylinder of equal height, minus a calibration value due to the ends. The diameter of a bubble can be approximated with the diameter of the glass tube. The length can be estimated with the width of the smallest enclosing rectangle when the entire bubble is visible, as shown in Figure 7. (The bubble is fully visible when there is sufficient distance between its border and the sections at which it enters and exits the field of view.) The calibration value can be calculated from reference measurements.

Lastly, two noise filtering techniques must be discussed. The experimental setup and the bubble location together could eliminate the majority of the environmental noises. However, in extreme cases, such as when the experimental setup was bumped into or when the pieces of black paper were moved during recording, the contours of the disturbances could be retrieved throughout the duration of the disruption. To prevent measuring errors, these were filtered in two stages. The contours with an insignificant area were discarded before the bubble tracking. Afterwards, every tracked object was assigned a lifetime value, which is the metric of how many frames the object has been tracked for. By selecting an appropriate lifetime threshold for measuring the volume, the short lived noises would disappear before mistakenly being assessed.

Implementation

The program itself consisted of two main phases: the initialization and the processing. The initialization phase included the selection of the source video, then the setup of the reference image, reference distance and the region of interest (ROI). These were kept as manual steps to simplify the code. Afterwards, the program processes each frame by locating, tracking and measuring the bubbles. When finished with the video, the application saves the results and exits.

The implementation used C++ using OpenCV and Visual Studio on Windows. To facilitate the development, multiple quality of life features were added. This includes a graphical user interface (GUI) with video controls and data visualization, a terminal display showing the calibration data and processing times, the ability to record the results in video and text formats, and a tool for creating and visualizing simple computer vision pipelines at runtime. To increase the performance, the graphical and terminal displays were managed by their own process. See Figure 8 for a snapshot of the GUI.

Testing

To assess the real-life usability of the application, both its real-time processing capability and its volume measuring accuracy had to be evaluated.

Real-time constraints

The real-time processing capability was defined in our case as the ability to determine the volume of passing gas within the examined tube section with negligible delay. This constraint can be fulfilled by either processing each frame before the next one becomes available or by being able to skip over frames without affecting the final result of the measurement.

To assess the performance of the application, a test consisting of processing videos with the region of interest limited to the tube was conducted. According to the logged time results, each frame of the video could be processed within a maximum of 12 milliseconds, which is within the available 20 millisecond time frame provided by the 50 fps videos. (For more details, see Figure 9 and Figure 10.) The test was run on a Lenovo W520 laptop equipped with an Nvidia Quadro 1000M GPU, 16GB of DDR3 RAM and an Intel i7-2670QM CPU.

In the tests afterwards, the robustness of the application was judged through its measuring precision when only every n-th frame is accessible. By carrying out the tests, it was found that processing only every n-th (n = 1 … 10) frame of the video had no significant effect on the measured volume. Thus the videos could be evaluated even with an effective refresh rate of 5 fps. (Note that the lifetime threshold for measuring the volume of a bubble had to be adjusted.) Bigger n-s were not tested.

Accuracy

To evaluate whether the application can be a substitute for the manual burette method, the measuring accuracy had to be determined for both. This required the consideration of what errors and to what degree affect the result and whether they are systematic or random.

The reference values were measured with an A class burette, calibrated at 20 ˚C, with a capacity of 10 ml, division of 0.1 ml and tolerance of ±0.03 ml. Despite the Schellbach stripe, the reading uncertainty was estimated to be ±0.03 ml. By evening out the fluid levels of the burette and of the other container, the read volume values could be used directly without any conversions. Given that the circulated deionised water was stored in an open tank, it could be assumed to be fully saturated with air. The setup and the substances were stored at 23 ˚C in the same room; as this likely caused the same error in the reference and in the video measurements, it was disregarded for both. After taking everything into consideration, the uncertainty of the reference burette measurements was evaluated to be ±0.06 ml.

Regarding the video measurement, several factors of uncertainty were collected. For most of them, however, the magnitude of uncertainty was deemed to be negligible. The distortion of the used camera lens was reported to be maximum 0.1% [21], [22], [23], [24] and thus there was no need for the camera calibration. The uncertainty of the reference lengths was minimized through selecting the two furthest marks of the ruler as reference points. The thin film of water between the bubbles and the glass tube was reported to be in the order of 4 μm for the equipment and materials used in the setup [15]; this value had an insignificant effect on the final measured volumes, and thus was discarded. For the same reason, the error of the glass tube diameter was also ignored. The uncertainty deriving from the pressure drop was also discarded, as neither the short tube section, the relatively low flow-rates, nor the horizontal tube placement would warrant pressures to significantly differ from atmospheric pressure. The parallax-error of the bubble-border determination was deemed to be negligible: based on evaluating the recordings, the lengths of the bubbles did not change significantly throughout their stay in view. Afterall, the primary source of uncertainty was deemed to originate from the - yet unaccounted - hemispherical ends of the bubbles.

To calibrate the length-based volume-measuring technique, the final recordings were split into calibration and test groups. The bubble-correction value for one video can be calculated from the uncalibrated measurement, the reference volume and the number of bubbles. (With the used channel diameter, both ends of a bubble were round; had the 2.8 mm tube been used, only the front would have been.) By doing so for each member of the calibration group, the average bubble-correction value can be determined. This enables the calibration of every measurement and the evaluation of the errors. The results are summarized in Figure 11 and Table 1.

Given the relatively low number of data points, the assessment of uncertainty is somewhat constrained. Certain techniques, such as calculating confidence intervals, identifying outlying data points and fitting trend lines or curves, produce results that lack statistical significance and therefore are not applicable. Nevertheless, the results show that the accuracy of the video based volume measurement is at most an order of magnitude below that of the manual burette method.

Conclusions

In my paper I discuss the development of an application that uses computer vision to measure the volume of passing gas in small-channel flow systems operating in the gas-liquid Taylor flow regime. The implementation and evaluation relied on videos and reference values taken with a self-assembled experimentation setup; its design is also discussed in detail.

Using C++ with OpenCV, the bubbles were located through grayscale image differencing, their contours were enhanced with thresholding and morphology, and they were tracked across consecutive frames using a spatial overlap based technique. The measurement of their volume assumed that a bubble can be approximated with a cylinder of equal length minus a correction value due to the hemispherical bubble ends. To acquire this value and to evaluate the measuring accuracy, the recordings were split into calibration and test groups. Using the members of the former, the average correction value was determined; this enabled the calibration of the measurements and the calculation of the errors.

Based on the conducted tests, the application is able to robustly process the videos in real-time. Through evaluating the measurements, the uncertainty of the volume measurement was evaluated to be at most one order of magnitude greater than that of the manual burette method. Nonetheless, further measurements would enable the determination of the accuracy with a greater certainty.

Future work

The presented application has the potential for further improvements and use cases. These are discussed in the following sections.

Experimental setup variations

The experimental setup circulated air and deionised water, and the recordings were taken of a circular glass tube with an inner diameter of 3.95 mm. The circulated substances were chosen due to their availability, non-toxicity and ease of handling, which enabled the conduction of experiments without any additional training or special equipment. The recordings were constrained to one tube diameter due to time limitations. Consequently, the practical usability of the application could be further evaluated if other gas-liquid substances common in flow systems and glass tubes of various diameters (and even of different cross sections) were also to be tested.

Improving the accuracy

The uncertainty of the video-based volume measurement was estimated to be at most one order of magnitude higher than that of the manual burette method. Nonetheless, this could be improved.

First of all, increasing the number of measurements would enable the calculation of confidence intervals, the identification of outlying data points and the analysis of the error distribution. These would support the evaluation of the measurement uncertainty with a higher confidence.

Additionally, the measurement itself could also be improved. The experimental setup had the shortcomings of being only able to produce bubbles with homogenous lengths and being able to measure only the total volume of the gas and not that of the individual bubbles. By injecting the air manually into the system with a syringe (as illustrated in Figure 12), it would be possible to create bubbles with the desired lengths or volumes. This, in turn, allows the evaluation of bubble attributes on a much wider scale, promoting the improvement of the volume-measuring technique.

Further uses

The application measured the volume of the passing gas within the flow system. However, with minor modifications it could be adapted for additional purposes.

First of all, using a reference image taken of the recording section filled with air - instead of water - would enable the location of the liquid slugs. Ultimately, this may enable the volume of the passing liquid to be measured in a similar manner as it was presented for the gas.

More importantly, however, locating the phase boundaries can also enable the evaluation of the bubble and slug lengths, the bubble velocity and the void fraction (the fraction of the flow domain occupied by the gas phase [14]) - with the same device. This could simplify the online monitoring of flow properties in small-channel flow systems operating in the gas-liquid Taylor flow regime.

References