This article highlights use cases of ocean observation to explore how cloud computing can be improved to handle increased data flows. As the amount of data ingested increases, the cloud could replace traditional approaches to data warehousing. High-performance mass storage of observational data, coupled with on-demand computing to run model simulations near the data, tools to manage workflows, and a framework to share and collaborate, enables a more flexible and adaptable observation and prediction computing architecture. Apply this structure in your industry regarding how to get data, store data, organize it, and conduct analysis and visualization in the cloud. What are some potential problems for large datasets? Think about how you would overcome those challenges. How would "sandboxes" provide some security when testing a system?
Preparing for the Next Waves of Users, Technologies, and Policies
The technical changes we have discussed are paralleled by human changes. In the past, marine research was mainly hands on – researchers went out and collected samples or measurements while on a cruise. As Kintisch has observed, graduate students are now
less likely to ever go to sea – they are working with satellite data, model outputs and aggregated datasets. Research groups have become virtual and an entire research program can be conducted in the cloud – from communication to gathering data to
analysis and the distribution of results. It behooves us to consider the human dimensions of cloud adoption as well as the technical strengths and weaknesses of the cloud.
As we look at the factors and variables affecting the move of organizations
to the Cloud for storage, resources, processing, and security, it is important to recognize the bi-directional impact of the next waves of users. Individual and institutional attitudes are shifting, prompting a new wave of Cloud users (Table 1). Many
have never known a professional or personal environment without the concept of being "online". They are mobile-savvy, always connected, hyper-aware of shifting technology trends and readily willing to adopt emerging technology. Organizations should
adopt best practices that reflect their work patterns and information needs, guaranteeing a much higher likelihood of their adoption and continuation of the data science integrity that is a core trait of the ocean science community.
TABLE 1
Wave | Connectivity |
Work patterns and needs |
Technology best practices that reflect their work patterns and information needs |
---|---|---|---|
The Future |
Always online, always connected |
Understand continuous information, less interested in data that produced the information Less likely to understand or be interested in the entire workflow rather they will focus on aspects of it: analysis, visualization relevance and context
of mashing up a variety of information sources. Challenge will be ensuring that the information they receive still supports deep science |
Expectation of immediacy and continuous connectivity to information Not tolerant of long processing times or difficulty in getting an answer to their question Demand speed of cloud or edge computing |
Digital Nomads |
Connect from anywhere, applications centric |
Understand continuous data collection, require information Comfortable with technology and view it as extension of themselves. Digital Nomads have a different relationship with information than technical or research users? |
May be frustrated by organizational inertia when it comes to adopting, leveraging and embracing the Cloud |
Technical Experts | Hard core large pipeline connectivity |
Understand entire workflow - sensor to results Want to know the details of what they are doing, are inclined to make updates and fixes themselves, and want to know the path of data from observation to use. They are very familiar with the conversion
of data to information and intimately understand every nuance of that conversion |
Well documented workflows in the cloud can enable them to share techniques and standardize paths. Need high capacity (cloud) computing resources and fast connectivity |
Researchers |
Lighter connectivity to the Internet |
Understand final results of data processing and expect quality data. Want to use data and need to be able to trust their quality but may not want to know all the details. They may be comfortable with a reliance on the automatic conversion
of data to information |
Need trusted data in the cloud and efficient, expandable and easily shared analysis and visualization tools Need high capacity (cloud) computing resources and fast connectivity |
Digitally divided | Limited connectivity - either permanently or situational |
Need information but may not have access to HPC etc. May be members of other waves working as first responders during disasters or at sea Need information and require technologies that will enable them to mitigate issues such as low
bandwidth or missing or destroyed communication infrastructure |
Need cloud-based technologies that support intermittent connectivity and asynchronous communication. Need trusted data in the cloud and efficient, expandable and easily shared analysis and visualization tools |
A Technological (R)evolution
A technology evolution is equally underway and it will dramatically affect ocean science and the Cloud. Autonomous vehicles, swarm robotics, Edge computing (aka sensor-based computing), in situ communications such as cabled observatories, and global availability
of low cost, high bandwidth communications will disrupt ocean data collection and distribution. Data will be processed at the source of collection, sensors and vehicles will autonomously make decisions based on that processed data, science-based machine
learning and the results will be broadcast in near-real time once the sensor is able to contact the Internet. Consumers will be a mix of human and machine end-points.
Future users will enjoy a near continuous Internet experience for non-submerged
devices and cabled observatories and the human and non-human consumption of the information that they generate. The emerging Ambient Internet will see the Internet effectively disappear as it becomes connected to nearly everything, and in particular,
devices, vehicles and sensors used for marine data collection. This will naturally lead to more data being collected. We are already nearing a tipping point of data volumes exceeding the human capacity to process it all. Automated and cloud-based
processing have been slow to materialize in this industry, and what is likely to happen is that it will be eclipsed by sensor-based processing.
Edge computing sits squarely in the realm of the Ambient Internet, leveraging machine learning,
artificial intelligence, advanced processing and computing power on the devices themselves and communicating securely and selectively with the Cloud for data transfer. Human reliance on the raw data itself will depreciate over time as the volume of
collected data becomes untenable. Confidence levels will increase as machine-learning algorithms consistently produce better results. There will be a nexus when confidence in machine based acquisition, processing and delivery of information exceeds
that of the human equivalent.
Tension Between the Current and the Future
As use of the Cloud becomes more widespread, a natural tension between users of local resources and cloud users will be created. Early adopters should feel compelled to prepare best practices to ease adoption for future waves of users. Some of these include
embracing the notion of true Digital Nomads. They will not be beholden to any particular platform, operating system, application or physical space. They will have never known an Internet that was not in their pocket, available to them at all times,
without constraint. Their expectations of immediacy will be unparalleled and is not centric to the data itself, but rather to the information that it possesses. The emerging Future wave is going to be comfortable with artificial intelligence and augmented
reality with a blurred line of information derived from humans or machines. They will live and work in a world of augmented intelligence, where artificial intelligence, machine learning and deep learning assist the human experience.
The behavioral
characteristics of the next wave(s), coupled with mainstream information technology and the inevitable reduction in cost and increase in proliferation of smart enabled marine based sensors, will cement the fundamental shift in data-information relationships.
Sensor based processing with results transmitted to the Cloud will force emerging ocean knowledge workers to have an information centric mindset rather than a data centric one. This mindset will make it easier for others to receive the information
they need without needing access to massive datasets. Cloud hosted weather models are a prime example where advances led by Technical Experts and Digital Nomads can benefit Researchers and the Digitally Divided.
Relevance to Cloud and Policy Today
Creation of the structures needed to support the work of all of the waves to do their jobs and advance ocean science is multi-faceted and need not be considered a monumental effort. Evolving and adapting the current mindsets around the Cloud, Edge computing,
artificial intelligence, machine learning and augmented reality will dramatically alter the landscape for future workers. Engaging with industry, both traditional and non-traditional, will help spur the innovation. It will also significantly enhance
the quantity and quality of data that is collected as the private sector works to produce more inexpensive and more capable devices that work in a connected world.
Data policies also need to shift. Following the old mantra of "collect it, process
it, publish it and store it" will not work in an environment of constantly updated information, huge data volumes and increased access through widespread and continuous Internet coverage. Although data security will remain highly important, there
will be demands to make data more available so non-human means can interrogate it, learn from it and apply those results to data banks of valuable information. These approaches need to percolate through all levels of organizations in order to create
a culture of innovation and preparedness.
Few would disagree that there is an enormous brain-trust resident in organizations all over the world. Intricate knowledge of data formats, sensor types, performance nuances, and metadata standards
(or lack thereof) are just some of the elements. There needs to be a concerted effort to increase documentation, standardization and openness in multiple areas in order to propagate and persist this knowledge.
New commercial opportunities may
develop that focus on supporting new waves of workers. Encouraging proposals, new grants, funding for innovation and joint partnerships that stimulate research, commercialization and productization of emerging technology are a beginning. Existing
companies also have an opportunity to embrace open data and standards, develop automated processing and better support Edge devices as they come online.
The transformation that is occurring does not just involve the Cloud. It is part of a larger
technology movement for smarter, smaller and more computing power all around us. The Cloud is only one piece of the transformation and remaining focused on the Cloud at the expense of Edge computing, smart devices, artificial intelligence, automated
processing and information centric workflows will not adequately prepare for the next waves of marine scientists. The combination of people, process, and technology – including the Cloud – must be interfaced effectively to develop ocean data and information
systems necessary to observe and predict our oceans, lakes and coasts of the future.