--> /* MENU */

Crystal - 0x Fog Computing

Dispersed computing service in fog provides flexible and dynamic decomposition of computing tasks. Unlike cloud-based distributed data processing frame- work, dispersed computing service in fog should tackle heterogeneity of fog nodes in terms of resource capacity and allocation problem.

Why Crystal Fog Computing Platform?



• Location transparency: Computing tasks should be able to run on any nodes without propagating actual location. This property allows us to slice computing task into smaller sub-tasks and execute them in remote fog nodes.

• Heterogeneity: Dynamic decomposition of computing task is necessary for fog where unequal type of resource nodes are available. Not only distributed data processing, but also general applications are able to be deployed in fog with task decomposition in the heterogeneity.

• Modularity: Our fog dispersed computing is designed as a collection of libraries such as MapReduce, video encoding, etc. The libraries are loosely-coupled each other with minimum core interfaces. Inter-layer or inter-module optimization is not applicable to solve this issue.

• Latency: Optimal location of computing nodes minimize computation latency by reducing time in data fetching and transmitting calculation results. Early calculation close to data source such as sensor or storage is a key property of fog computing.

• Automation: Due to high complexity in configuration and resource selection, developers cannot handle all deployment possibilities to find optimal performance. To avoid the exhaustive task, fog computing service encapsu- lates deployment complexity by automating code slicing, resource selection, and deployment with minimum intention from developers.

0x Crystal Technology

Core Features

Developing a fault-tolerant fog application spanning over fog nodes requires high programming complexity and dealing with all the exceptions and failures, in- evitably leading to complicated application development process. To tackle these issues, the proposed fog computing framework Crystal provides an easy abstrac- tion for fog application development. The name Crystal is derived from ice fog, a type of fog composed of tiny ice crystals suspended in the air. Similarly, in Crystal framework, a fog application is composed of multiple components called Crystals spanning over fog nodes. Crystals are loosely-coupled components, each of which can run a standalone component, and they communicate each other through mes- sage passing. The functional definition of a Crystal component can vary from function-level to microservice-level depending on its roles in a fog application.

The proposed Crystal framework follows ’let it disappear’ philosophy to re- flect the inherently unreliable nature of fog where any node can leave and join at any moment. Whenever a fog node disappears from the fog by any reason, a fog application consisting of Crystals automatically heals by exploiting available fog and/or cloud nodes. Thus, developers can use Crystals as building blocks for fog applications without concerning failures of fog nodes. A fog application using Crystals can take full advantage of location transparency, self-healing, auto- scaling and mobility support described below:



Crystal: 0xPlatform's Next-Generation Fog Computing Platform

Location Transparency

Crystal components should be able to run on any node at any time. While the location of each Crystal instance may affect the overall performance of a fog application, it should not affect the context of the application. The concept of Crystal’s location transparency enables a fog application to be easily decomposed and deployed over a pool of fog nodes. Also, location transparency gives developers an illusion that a fog application development is more like a local application development.

Self-Healing

Crystal components should embed a self-healing functionality to survive in fog environment. Fog’s highly unreliable and unpredictable nature imposes a significant burden on developers to deal with all the exception and error handlings. Crystal assumes that any node can fail at any moment in fog. In that sense, Crystal components keep monitoring and respawning each other for self- healing, rather than expecting developers to program an error-free fog application. A fog application can be presented as a directed acyclic graph according to the communication patterns of its Crystals. When a parent Crystal detects a failure from its child Crystal instance, it automatically re-creates it on another fog node. All of these procedures are transparent to the user application, and also to the user code.

Automatic Scaling

Crystal components should automatically scale out and scale in when a Crystal instance is busy or idle, respectively. In case of a stateless Crystal instance, Crystals are seamlessly scaled out and scaled in by creating or killing replicated Crystal instances. Messages to the replicated Crystal instances are automatically load-balanced. In case of a stateful Crystal instance, its state is written to a distributed data store and accessed by replicated Crystal instances.

Mobility Support

Crystal components should move from one fog node to another when migration is required. With Crystal’s mobility, a fog application consisting of Crystals can dynamically expand itself from things, to the fog, to the cloud, and also vice versa. When a Crystal instance of a fog application does not meet the minimum running requirements anymore (e.g., lack of resources, or location change), it finds an adequate fog node and migrates to the node. Crystals are responsible for delivering the messages designated to the migrated Crystal instance and it is transparent to the user code.

0x Crystal Technology

System Architecture

This figure shows the overall architecture of Crystal-based fog computing. When a fog application is initiated, it is decomposed into subtasks by the fog agent. The fog agent then transforms the subtasks into standalone and deployable Crystal instances. To deploy these Crystal instances, the fog agent first sends a resource request to a fog tracker which tracks the list of fog nodes and their resource availability. According to each Crystal instance’s resource requirements, the fog tracker sends back the list of most adequate fog nodes.



Software Stack

Software stacks of the proposed distributed Fog computing platform is shown in the figure below. Crystal framework is made up of three main components: Fog Agent, Fog Tracker and Fog Programming Interfaces.



Crystal Programming APIs

OPERATION DESCRIPTION
send Send a message to a Crystal
receive Receive a message and process it
stayInCloud Crystal will stay in the cloud
stayInFog Crystal will stay in the local fog cluster
stayInThing Crystal will stay in the client node where it was created
moveToCloud Crystal will move to the cloud
moveToFog Crystal will move to the fog
moveToThing Crystal will move to the client node
replicateCrystal Scale out a Crystal and do load balancing
spawnCrystal Create a Crystal instance on a node
migrateCrystal Migrate a Crystal instance to a node
checkpointCrystal Save the state of Crystal
respawnCrystal Re-create a Crystal from a checkpoint
killCrystal Terminate Crystal instance
getCrystal Get name of Crystal
getCrystalList Get the list of Crystals for an application
getCrystalParent Get the list of Crystal’s parent Crystals
getCrystalChild Get the list of Crystal’s child Crystals
getCrystalLocation Get location information of a Crystal
getCrystalLatency Get latency from a Crystal to another
getNode Get name of a fog node
getNodeList Get the list of fog nodes in the fog cluster
getNodeLocation Get location information of a node
getNodeLatency Get latency from Crystal to a fog node
getFog Get name of the local fog cluster
getFogList Get the list of fog clusters
getFogLocation Get location coverage of a fog cluster
getFogLatency Get latency from Crystal to a fog cluster
setLatency Set latency limit a Crystal has to guarantee
setStorage Set storage limit a Crystal can use
setMemory Set memory limit a Crystal can use
setCPU Set CPU limit a Crystal can use

Crystal Event Handlers

EVENT DESCRIPTION
onCreate New Crystal created
onFailure Crystal failed
onKill Crystal killed
onMigrate Crystal migrated
onReplicate Crystal replicated
onHighLatency Average Latency is higher than limit
onLowCPU Node running out of CPU
onLowMemory Node running out of memory
onLowStorage Node running out of storage