LizardFS 3.12.0 RC is out!

Release Candidate of 3.12.0

Please test it and let us know about any issues.

Description below.

Featuring:

 

– C API

– nfs-ganesha plugin

– RichACL – a new POSIX + NFSv4 compatible ACL standard

– OSX ACL support through osxfuse

– ACL in-memory deduplication

– file lock fixes

– AVX2 support for erasure code goals

– MinGW compilation fixes

– more flexible chunkserver options

– many fixes

Detailed info:

* C API *

LizardFS 3.12 comes with liblizardfs-client library and C language API header.

It’s now possible to build programs/plugins with direct support for LizardFS operations,

no FUSE needed. For reference, see:

src/mount/client/lizardfs_c_api.h

src/data/liblizardfs-client-example.c

 

For those building LizardFS from source, pass a -DENABLE_CLIENT_LIB=YES flag to cmake

in order to make sure you’re building client library as well.

* nfs-ganesha plugin *

 

Our official plugin for Ganesha NFS server is included as well. This plugin enables

a LizardFS FSAL (File System Abstraction Layer) to Ganesha, which is then used

to access LizardFS clusters directly. Our new plugin is pNFS and NFSv4.1 friendly.

 

For those building LizardFS from source, pass a -DENABLE_NFS_GANESHA=YES flag to cmake in order to make sure you’re building client library as well.

* RichACL *

In order to extend POSIX access control list implementation we introduced RichACL support.

Backward compatibility with POSIX ACLs is guaranteed. Additionally, it’s possible to use NFSv4-style ACL tools (nfs4_getfacl/nfs4_setfacl) and RichACL tools (getrichacl/setrichacl) to manage more complicated access control rules.

* OSX ACL *

Setting/getting ACLs is also possible on OSX via both command line chmod/ls -e interface and desktop.

 

* File lock fixes *

Global file locking mechanism is now fully fixed and passes all NFS lock tests from connectathon suite.

 

* AVX2 *

Erasure code goal computing routines now take full advantage of AVX2 processor extensions.

 

* MinGW *

LizardFS is now bug-free again for MinGW cross-compiling.

 

* Chunkserver options *

Replication limits are now fully configurable in chunkserver config.

Also, chunk test (a.k.a. scrubbing) has 1 millisecond precision now instead of previous 1 second, which allows users to turn on more aggressive scrubbing with simple chunkserver reload.

 

https://www.google.com/url?q=https%3A%2F%2Fgithub.com%2Flizardfs%2Flizardfs%2Freleases%2Ftag%2Fv3.12.0-rc1&sa=D&sntz=1&usg=AFQjCNFouUJf1ZZkQPypA9rHJHNpXxvKJA

 

LizardFS on OpenNebula conference

Michal Bielicki showing how to connect 1,9 PB cluster to Nodeweaver.

Follow the link to see the full video.

 

OLYMPUS DIGITAL CAMERA

 

New LizardFS Brochure

LizardFS in a nutshell.

Check out our new brochure, highlighting features, benefits and possible use cases!

LizardFS Brochure

 

 

LizardFS Expands Global Presence in United States with Enterprise Support

We are pleased to welcome Michael Kozlowski on the team who will be responsible for scaling the sales team and growing markets across the United States.

With his extensive experience we are sure that he will make a huge impact.

We are excited to work with you Mike!

Read more:

http://www.pr.com/press-release/732363

 

Come meet us at the Wolves Summit

Time flies… We took part in 5th edition and the 6th edition of Wolves Summit is almost here, it will take place on 10-11 October 2017 in Warsaw.

LizardFS will be there sharing our experiences with the startup community and SMBs. Looking forward to meeting potential clients, partners and visionaries! Hope this one will be just as inspiring as the previous one.

Let us know if you will be attending too, always happy to have a chat.

 

LizardFS as The Storage Solution for SAS Analytics

Nothing gives you more satisfaction than a happy customer. Well… Maybe a happy customer that also makes you aware of a niche in which your product performs extremely well.

More details will come soon with some hard numbers however we can already share some initial insights.

Business users are delighted with the performance which isn’t dropping even with many concurrent clients connected to a 500TB cluster on 6 nodes.

Another important factor that’s been appreciated is its stability, LizardFS is pretty much maintenance-free.

 

How To Change The World on a Budget

 

LIFE SCIENCES & OPEN SOURCE STORAGE

The scientific and research community tends to be a budget and grant driven environment, and the need to scale storage can often be sudden and unexpected. Traditional appliance based scale out storage vendors often take advantage of customers predicaments by charging expansion prices far in excess of what it costs to buy the solution initially. This forces customers to either accept the ransom, or have to seek out alternative storage solutions that do not integrate with their existing infrastructure. This not only increases the data management costs but also leaves them with complex and disjointed storage infrastructures.

 

With most Open Source Software Defined Storage solutions users are protected from vendor lock in. Organisations are free to choose the most cost efficient storage hardware from the vendor of their choice. By combining vendor-agnostic disk, tape, flash, ssd and cloud storage technologies into a single unified data repository, unpredictable storage costs, ‘forklift’ upgrades and unmanageable disjointed infrastructures of storage technology are completely eliminated. While your storage costs are decreasing by up to 90%.

Open Source solutions can provide a greater return on your storage investment by opening the path to competitive technology procurement. Automated data management unifies the end-to-end storage footprint and enables our users to spend a far greater proportion of their time and budgets on accelerating science and research (changing the world).

LizardFS is an example of Open Source Software-Defined Storage. It provides enterprise-class storage using commodity hardware and specialized software to deliver storage services, advanced features, and management capabilities. Compared to traditional enterprise storage that requires proprietary or custom storage systems, open source software-defined storage platforms such as LizardFS have much lower up-front and ongoing operational costs.

This solution removes the complexity that historically has burdened organizations who struggle to keep up with massive data growth, and simplifies the storage processes for IT professionals in all types of industries. LizardFS increases IT agility by enabling organizations to leverage hardware from any vendor, as well as the option to use existing IT infrastructure to create a custom storage solution to any storage need. It adjusts well to new technologies giving the opportunity to tune and increase performance by integrating ssd/ flash / NvME etc. Organizations can achieve massive scale by increasing storage capacity and performance as needed up to 8192 chunkservers and 1 Exabyte of data.

The life sciences industry, is struggling to keep up with massive data growth due to advancing technologies such as genomics. The processing of data from one genome produces about 1.5 gigabytes, that in turn creates huge amounts of storage needs for each genomics organization that stores thousands of sequenced genomes each day. Genome researchers and other life sciences professionals require storage systems that are high-performing, secure, and scalable. The processing of large datasets also requires fast storage that can concurrently handle many simultaneous write streams as data is processed. Below is a typical data flow involving LizardFS.

LizardFS has had many success stories within the Genomics sphere, some examples:

 

 

Aalborg University Hospital (Aalborg, Denmark) have been using LizardFS for over a year now. They store human exome sequencing data, currently they have 80TB of raw data stored and are adding 80-100GB per week. Vang Qu Le, the bioinformatician in charge of the storage facility was tasked with finding a replacement for the the existing NFS setup. As mentioned earlier, they are bound with the same budget restrictions like most of the scientific and research community, so the replacement had to be scalable but cheap . “We have existing workstations, with internal 10TB storage, and other hardware, we need to make use of them, for cost effectiveness.” Since installation and familiarization with the system he has had no issues with projects or meeting deadlines.

 

United States Department of Agriculture (USDA), they are actually in the process of setting up their cluster as we speak, but intend to be using it for 1PB of sequenced swine data, their use case is somewhat different from Vang’s, in that their primary concern is hurricanes, so they are looking to use LizardFS to replicate data between 2 different data centers in different parts of the state, just in case one of them gets taken off to OZ, they will also hopefully be making use of some of the other features that LizardFS has to offer, like erasure coding to reduce the amount of space their pigs take up in the data centers.

 

Another user of LizardFS is Brent Matthews, the linux administrator at Complete Genomics based in California and owned by BGI the world’s largest Genomics services company. They are doing WGS (whole genome sequencing) for them each genome can consume from 500gb up to 1.5TB of storage. They have been using LizardFS for sometime now and has a 22 node cluster of 3.8PB which is 80% full. Brent looked at several alternatives to LizardFS, but finally settled on it as it was so easy to set up, scalable, cost effective and reliability would not need to be an issue anymore. Brent loves that he does not have to babysit the system anymore, it just works!

We have set ourselves a clear goal to deliver the best software defined storage solution on the market. However, we know that even when we reach our current objective, we will not stop innovating.

Some examples of our latest innovations that are being tested and about to be launched for LizardFS, are a Hadoop plugin to allow you to connect your LizardFS storage to a Hadoop cluster and NFS 4.1 giving you full support for pNFS.

So if you are in the business of trying to make the world a better place, you might like to get in touch and discuss more details of how LizardFS can help you achieve your world changing goals.

 

Mark Mulrainey

Storage Dilemma Solver

+48 733 187 097

mark.mulrainey@lizardfs.com

https://lizardfs.com/

 

LizardFS available in Fedora repositories.

One of the biggest advantages of being Open Source is the help and the support that we are getting from random people.

Thanks to Jonathan Dieter LizardFS has found a place in the official Fedora repositories!

Try out this nice mixture yourself.

https://admin.fedoraproject.org/pkgdb/package/rpms/lizardfs/

Once again – big thank you to Jonathan.

 

Tech Retreat – HPA Oxford

We will be attending the Hollywood Professionals Association Tech Retreat in Oxford. Excited to see what is new in the world of media.

Looking forward to meeting all the media engineers interested in data storage!

 

Scale Out Storage on OpenNebula Conference.

LizardFS will have a hands on session at the OpenNebula conference.

You are invited to join us in Madrid 23-24 of October to have some fun with storage clusters.

This also means that further integration with OpenNebula is something that we are currently working on.

More details about the workshop and news about the integration progress to come soon, so stay tuned and follow us on Facebook, LinkedIn and Twitter!

Hope to see you in Spain!