Popular Backup/Archival Service and its Application for the Archival of the Network Traffic in the PIONIER Academic Network
Brzeźniak Maciej, Meyer Norbert, Mikołajczak Rafał, Jankowski Gracjan, Jankowski Michał
Poznań Supercomputing and Networking Center
Institute of Bioorganic Chemistry, Polish Academy of Sciences
Noskowskiego 12/14, 61-704 Poznań, Poland
e-mail: {maciekb/meyer/rafal.mikolajczak/gracjan/jankowsk}@man.poznan.pl
Received:
Received: 27 September 2010; revised: 8 November 2010; published online: 23 November 2010
DOI: 10.12921/cmst.2010.SI.01.109-118
OAI: oai:lib.psnc.pl:679
Abstract:
This paper presents the popular backup/archival service developed and operated in Poland by members of the PIONIER network consortium and its example application for outsourcing of the archival of the network traffic in the national academic network. The service is built upon the National Data Storage (NDS) system architecture deployed in the redundant, high-end, geographically distributed infrastructure of servers, network and data storage systems built within the confines of the PLATON project. The details of the NDS architecture and its features are discussed in the paper including the system components, their functionality and the system
scalability aspects. The paper also presents how the NDS architecture is deployed in the data storage infrastructure of the PLATON project, with an extensive usage of servers and storage virtualization technologies. We discuss how the NDS system instantiation allows for flexible set up of the multiple instances of the popular backup/archival service, which can address various, often contradictory requirements of the service users, while sharing a common pool of physical resources. As an example the system set up for outsourcing the archival of the PIONIER network traffic is presented
Key words:
backup, data archival, data management, distributed data storage, virtualization
References:
[IDC1] IDC analysis. Cited among others in: Humans created 161 exabytes of data in 2006.
http://www.itnews.com.au/News/74870,humanscreated- 161-exabytes-of-data-in-2006.aspx [PIO1] PIONIER – Polish Optical Internet – a nationwide broadband optical network for e-science. http://www.pionier.net.pl/online/en/projects/69/PIONI ER_Network.html
[NDS1] National Data Storage project in Poland. Project Web page: nds.psnc.pl
[GUS1] Source: polish Central Statistics Office. http://www.stat.gov.pl/gus/index_ENG_HTML.htm
[MIN1] Source: polish Central Statistics Office. http://www.stat.gov.pl/gus/index_ENG_HTML.htm, cited by:
http://www.studenckamarka.pl/serwis.php?s=73&pok =1909
[FUSE1] Filesystem in Userspace. http://fuse.sourceforge.net
[PSQ1] PostgreSQL. The world’s most advanced open source database. http://www.postgresql.org
[SLO1] Slony-I. Enterprise-level replication system. http://www.slony.info/
[VMW1] VMware vSphere Hypervisor (ESXi). http://www.vmware.com/products/vspherehypervisor/ index.html
[HSM1] HSM. TSM-HSM, Tivoli Storage Manager for Space Management.
http://www-306.ibm.com/software/tivoli/products/ storage-mgr-space/
[DCA1] http://www.dcache.org
[DCA2] P. Millar, dCache. Presentation during 3rd Terena TFStorage Meeting, Dublin, 2009.
http://www.terena.org/activities/tfstorage/ ws5/agenda.html
[DCA3] P. Fuhrmann, V. Gulzow, dCache, storage system for the future, In: W. E. Nagel, W. V. Walter, W. Lehner (Eds.): Euro-Par 2006, Parallel Processing, 12th International Euro-Par Conference, Dresden, Germany, August 28-September 1, 2006, Proceedings. LNCS 4128, Springer 2006,
[CHI1] Chimera – a new, fast, extensible and Grid enabled namespace service,
http://www.dcache.org/manuals/chep06/Chimerapaper- chep06.pdf
[IRO1] https://www.irods.org
[IRO2] Arcot Rajasekar, Mike Wan, Reagan Moore, Wayne Schroeder, A Prototype Rule-based Distributed Data Management System HPDC workshop on “Next Generation Distributed Data Management”, Paris 2006.
This paper presents the popular backup/archival service developed and operated in Poland by members of the PIONIER network consortium and its example application for outsourcing of the archival of the network traffic in the national academic network. The service is built upon the National Data Storage (NDS) system architecture deployed in the redundant, high-end, geographically distributed infrastructure of servers, network and data storage systems built within the confines of the PLATON project. The details of the NDS architecture and its features are discussed in the paper including the system components, their functionality and the system
scalability aspects. The paper also presents how the NDS architecture is deployed in the data storage infrastructure of the PLATON project, with an extensive usage of servers and storage virtualization technologies. We discuss how the NDS system instantiation allows for flexible set up of the multiple instances of the popular backup/archival service, which can address various, often contradictory requirements of the service users, while sharing a common pool of physical resources. As an example the system set up for outsourcing the archival of the PIONIER network traffic is presented
Key words:
backup, data archival, data management, distributed data storage, virtualization
References:
[IDC1] IDC analysis. Cited among others in: Humans created 161 exabytes of data in 2006.
http://www.itnews.com.au/News/74870,humanscreated- 161-exabytes-of-data-in-2006.aspx [PIO1] PIONIER – Polish Optical Internet – a nationwide broadband optical network for e-science. http://www.pionier.net.pl/online/en/projects/69/PIONI ER_Network.html
[NDS1] National Data Storage project in Poland. Project Web page: nds.psnc.pl
[GUS1] Source: polish Central Statistics Office. http://www.stat.gov.pl/gus/index_ENG_HTML.htm
[MIN1] Source: polish Central Statistics Office. http://www.stat.gov.pl/gus/index_ENG_HTML.htm, cited by:
http://www.studenckamarka.pl/serwis.php?s=73&pok =1909
[FUSE1] Filesystem in Userspace. http://fuse.sourceforge.net
[PSQ1] PostgreSQL. The world’s most advanced open source database. http://www.postgresql.org
[SLO1] Slony-I. Enterprise-level replication system. http://www.slony.info/
[VMW1] VMware vSphere Hypervisor (ESXi). http://www.vmware.com/products/vspherehypervisor/ index.html
[HSM1] HSM. TSM-HSM, Tivoli Storage Manager for Space Management.
http://www-306.ibm.com/software/tivoli/products/ storage-mgr-space/
[DCA1] http://www.dcache.org
[DCA2] P. Millar, dCache. Presentation during 3rd Terena TFStorage Meeting, Dublin, 2009.
http://www.terena.org/activities/tfstorage/ ws5/agenda.html
[DCA3] P. Fuhrmann, V. Gulzow, dCache, storage system for the future, In: W. E. Nagel, W. V. Walter, W. Lehner (Eds.): Euro-Par 2006, Parallel Processing, 12th International Euro-Par Conference, Dresden, Germany, August 28-September 1, 2006, Proceedings. LNCS 4128, Springer 2006,
[CHI1] Chimera – a new, fast, extensible and Grid enabled namespace service,
http://www.dcache.org/manuals/chep06/Chimerapaper- chep06.pdf
[IRO1] https://www.irods.org
[IRO2] Arcot Rajasekar, Mike Wan, Reagan Moore, Wayne Schroeder, A Prototype Rule-based Distributed Data Management System HPDC workshop on “Next Generation Distributed Data Management”, Paris 2006.
