Fast and scalable ownCloud Infinite Scale

Everything Must Go

Let's Go

The programming language Go brings in a large number of advantages (e.g., concurrency). Most computer programs, even if they appear to be one monolithic piece of code from the outside, are split into different components internally. Web servers (e.g., the Apache web server, which is usually deployed with ownCloud) are excellent examples. Internally, they usually have a function handling TCP/IP connections; another function might be responsible for handling SSL, and yet another piece of code might execute the requested PHP files and deliver the results to the end user.

All of those events happen, however, in a certain order. PHP and Apache simply can't handle concurrency: The individual steps of handling requests are done one after another, not at the same time, even if it was logically possible. For a client request to be served, multiple steps must be performed. Any environment with concurrency will allow all those events to happen at the same time instead of serially. Servers that are capable of handling requests in parallel will have to wait less for processes to finish their work, thus offering faster results to the user. By the way, this feature is one of the reasons why Go has become one of the de facto standards for containerized microarchitecture applications.

A More Modular ownCloud

ownCloud is adapting to an architecture centered around the principle of microservices. oCIS splits into three tiers: storage, core, and front end. In this article, we look at each of these tiers separately. Even though for clients who upload and download files all that counts is the overall performance (including bandwidth), choosing the right underlying storage is crucial. In its three-tier model, ownCloud considers the storage available to the system to be the lowest tier.

Along with performance goes scalability: Large ownCloud instances must be able to cope with the load of thousands of clients and must allow additional disk space if the existing space fills up. Like so many concepts, object stores and scalable storage wasn't as available in the past when ownCloud was initially designed as it is today, which is yet another good argument to review oCIS.

Administrators today are used to having more choices, and ownCloud permits outsourcing the handling of physical storage devices to an external solution. Although S3-based object storage, storage based on the Samba service, or POSIX-compatible filesystem options continue to be supported, the preferred way to deploy oCIS is the way of Earth Observing System (EOS) Storage [3].

EOS to the Rescue

Originally provided by CERN in Geneva for its Large Hadron Collider experiment, EOS is optimized to feature very low latency when accessing files. It provides disk-based storage to clients by the XRootD [4] framework but also permits other protocols to access files. ownCloud uses the EOS HTTP protocol extension to talk to the storage solution (over HTTPS, of course). Far more interesting, though, is that it allows almost infinite scalability. For instance, at the time of writing this article, the CERN setup featured more than 200PB of disk storage and continues to grow. By turning to EOS, ownCloud got rid of a number of shortcomings: EOS does not have a typical single point of failure, and all relevant services are run redundantly, including its ability to scale out and add additional instances of all existing services. EOS promises never to run out of disk space and comes with built-in redundancy for the data stored therein.

Buy this article as PDF

Express-Checkout as PDF
Price $2.95
(incl. VAT)

Buy ADMIN Magazine

SINGLE ISSUES
 
SUBSCRIPTIONS
 
TABLET & SMARTPHONE APPS
Get it on Google Play

US / Canada

Get it on Google Play

UK / Australia

Related content

  • ownCloud

    ownCloud is an alternative to the many online storage services whose data protection promises could turn out to be as nebulous as their offerings.

  • Ubuntu 16.04 LTS as an ownCloud server
    Locally hosted collaboration servers are an alternative to letting employees swap files in a public cloud. We show you how to set up an ownCloud 9 environment on Ubuntu 16.04 LTS.
  • Spotlight on the Kubernetes package manager, Helm
    A Helm chart is a template of several parts that defines, deploys, and upgrades Kubernetes apps and can be considered the standard package manager in the Kubernetes world.
  • Nine home clouds compared
    Dropbox was the first of a number of cloud service providers. However, only services that promise full control over your own data can give users a feeling of security. We provide an overview of nine cloud projects and two BitTorrent tools.
  • Identity and access management with Authelia
    Add access controls to web applications that do not have their own user administration; however, this useful gatekeeper requires a reverse proxy.
comments powered by Disqus