Introduction to ZooKeeper

Introduction to Distributed System

Many independent programs running on set of different computers.

Introduction to Coordination Service

Co-ordination service is service which

  1. communicated with all computers in system.
  2. coordinates with all computers in system.
  3. manages all computers in system.
  4. controls with all computers in system.

What is ZooKeeper

Apache ZooKeeper is the coordination service for distribute systems. It is

  • Open source
  • Reliable & scalable
  • High-performance
  • Lightweight

Distributed System

Understanding distributed system

A distributed system is defined as a software system that is composed of independent computing entities linked together by a computer network whose components communicate and coordinate with each other to achieve a common goal.

Characteristics of distributed system

  1. Resource sharing(资源贡献):
    This refers to the possibility of using the resources in the system, such as storage space, computing power, data and services from anywhere and so on…
  2. Extensibility(可扩展):
    This refers to the possibility of extending & improving the system incrementally, both from hardware and software perspectives.
  3. Concurrency(并发):
    This refers to the system’s capability to be used by multiple users at the same time to accomplish the same task or different tasks.
  4. Performance and scalability(性能和可扩展性):
    This ensure that the response time of the system doesn’t degrade as the overall load increases.
  5. Fault tolerance(容错):
    This ensures that the system is always available even if some of the components fail or operate in a degraded mode.
  6. Abstraction through APIs:
    This ensures that system’s individual components are concealed from the end users, revealing only the end services to them.

Fallacies while designing Network File System

Perfect Assumptions

  • The network is reliable
  • Latency is zero
  • Bandwidth is infinite
  • The network is secure
  • Topology doesn’t change(拓扑不变)
  • There is one administrator
  • Transport cost is zero
  • The network is homogeneous

Challenges while implementing coordination in distributed system

  • Everything depends on master node. Dependence (The master is offline, then all clients belong it is unavailable.)
  • Service Discovery
  1. To increase availability of application & bear the load, we add more servers.
  2. We can tell client machines about availability of new servers.
  3. To do this, we need to implement that logic in client carefully.
  • Scalability
  1. Scalability means to make system grow bigger in size.
  2. By adding new machines, chances of crashing the system also increases.
  3. Common reasons of failure are hardware faults, system crashes, communication link failures

Introduction to Apache ZooKeeper

  • Apache Zookeeper is a software developed by Apache.
  • It acts as centralized service & maintains naming & configuration data.
  • It also provides flexible and robust synchronization within distributed systems.

Architectural Services of ZooKeeper

  1. Distributed Consensus:
    Its an agreement on a single data value among a group of process connected by an unreliable network.
  2. Group Management:
    Managing groups of nodes in a distributed environment always requires a critical implementation of high-performance coordination services.

  3. Presence Protocol:
    This defines presence-related extensions session initiation. This set of protocols describe how other protocols are used by terminals to establish, modify, and terminate sessions.

  4. Leader Election:
    Leader election is the process of designating a single process as the organizer of some task distributed among several nodes

ZooKeeper coordination tasks

With ZooKeeper, developers can implement common distributed coordination tasks, such as the following:

  1. Configuration management
  2. Naming service
  3. Distributed synchronization, such as locks and barriers
  4. Cluster membership operations, such as detection of node leave/node join

ZooKeeper Ensemble

In a production environment, ZooKeeper should be run on multiple servers in a replicated mode, also called a ZooKeeper ensemble.

The minimum recommended number of servers is three, and five is the most common in a production environment.

Chapter 2

Understanding ZooKeeper Services

  • ZooKeeper is coordination service for distributed applications.

  • So its objective is to solve the difficult issues associated with the coordination of components in a distributed application.

  • This is done by exposing simple powerful interface of primitives.

  • Applications can be designed on these primitives implemented through ZooKeeper APIs.
    可以在通过ZooKeeper API实现的这些原语上设计应用程序。

  • It can solve problems of

    • distributed synchronization
    • cluster configuration management
    • group membership and so on.
      它可以解决 分布式同步 集群配置管理 小组成员等等。
  • ZooKeeper is a replicated and distributed application with the intention to run as a service.

  • This replicated set of servers on which the ZooKeeper service is running, called ZooKeeper ensemble.

  • Clients can connect to a ZooKeeper service by connecting to any member of the ensemble.

  • The members of the ensemble are aware of each other’s state.

  • They save information in durable manner in the local data store.

  • ZooKeeper is a highly available service

  • As long as a majority of the servers are available, the service will always be available.

ZooKeeper Data Model

  • ZooKeeper allows distributed processes to coordinate with each other through a shared hierarchical namespace of data registers.

  • The namespace looks quite similar to a Unix file system.

  • The data registers are known as znodes in the ZooKeeper.

  • znodes are organized hierarchically as a standard file system.

  • The root node has one child znode called /zoo.

  • Every znode in the ZooKeeper tree is identified by a path.

  • The path elements are separated by /.

  • znodes are called data registers as they can store data.

  • The znode can have children as well as data associated with it.

  • Every znode maintains a stat structure.

  • A stat simply provides the metadata of a znode.
    It consists of

    • Version number
    • Action Control List (ACL)
    • Data length
    • Timestamp
  • Version number:
    Every znode has a version number and it means every time the data associated with the znode changes, its version number would also increase.
    The use of version number is important when multiple zookeeper clients are trying to perform operations over the same znode.

  • Action Control List (ACL):
    It is authentication mechanism for accessing the znode.
    It governs all the znode read and write operations.

  • Data length:
    It is total amount of the data stored in a znode.
    You can store a maximum of 1MB of data.

  • Timestamp:
    It is time elapsed from znode creation & modification. It is usually represented in milliseconds.
    ZooKeeper identifies every change to the znodes from “Transaction ID” (zxid).
    Zxid is unique & maintains time for each transaction so that you can easily identify the time elapsed from one request to another request.

Have fun.