分布式文件系统(Distributed File System)是指文件系统管理的物理存储资源不一定直接连接在本地节点上,而是通过计算机网络与节点相连。分布式文件系统的设计基于客户机/服务器模式。一个典型的网络可能包括多个供多用户访问的服务器。另外,对等特性允许一些系统扮演客户机和服务器的双重角色。
FastDFS是一个开源的轻量级分布式文件系统,它对文件进行管理,功能包括:文件存储、文件同步、文件访问(文件上传、文件下载)等,解决了大容量存储和负载均衡的问题。特别适合以文件为载体的在线服务,如相册网站、视频网站等等。
一:安装FastDFS
1、首先安装libevent。
- wget http://monkey.org/~provos/libevent-2.0.11-stable.tar.gz
- tar -zxvf libevent-2.0.11-stable.tar.gz
- cd libevent-2.0.11-stable
- ./configure --prefix=/usr
- make clean
- make
- make install
2、安装FastDFS的步骤
- wget http://fastdfs.googlecode.com/files/FastDFS_v3.03.tar.gz
- tar -zxvf FastDFS_v3.03.tar.gz
- cd FastDFS
- vim make.sh 将#WITH_HTTPD=1 修改成WITH_HTTPD=1 以支持 http
- ./make.sh
- ./make.sh install
二:FastDFS的配置
1、配置及启动Tracker Server:
- mkdir /home/fastdfs
- vi /etc/fdfs/tracker.conf
- base_path=/home/yuqing/fastdfs -> base_path=/home/fastdfs
- http.server_port=8080 -> http.server_port=9090
- ##include http.conf -> #include http.conf
- reserved_storage_space = 4GB -> reserved_storage_space = 1GB
- cd /usr/local/bin
- fdfs_trackerd /etc/fdfs/tracker.conf
检查tracker是否启动成功,可以查看如下文件/home/fastdfs/logs/trackerd.log:
- [2011-10-21 14:29:44] INFO - FastDFS v3.03, base_path=/home/fastdfs, run_by_group=, run_by_user=, connect_timeout=30s, network_timeout=60s, port=22122, bind_addr=, max_connections=256, work_threads=4, store_lookup=2, store_group=, store_server=0, store_path=0, reserved_storage_space=1024MB, download_server=0, allow_ip_count=-1, sync_log_buff_interval=10s, check_active_interval=120s, thread_stack_size=64 KB, storage_ip_changed_auto_adjust=1, storage_sync_file_max_delay=86400s, storage_sync_file_max_time=300s, use_trunk_file=0, slot_min_size=256, slot_max_size=16 MB, trunk_file_size=64 MB
- [2011-10-21 14:29:44] INFO - HTTP supported: server_port=9090, default_content_type=application/octet-stream, anti_steal_token=0, token_ttl=0s, anti_steal_secret_key length=0, token_check_fail content_type=, token_check_fail buff length=0, check_active_interval=30, check_active_type=tcp, check_active_uri=/status.html
2、配置及启动Storage Server:
- mkdir /home/fastdfs/fdfs_storage
- cd /etc/fdfs
- vi storage.conf
- base_path=/home/yuqing/fastdfs -> /home/fastdfs/fdfs_storage
- store_path0=/home/yuqing/fastdfs -> store_path0=/home/fastdfs/fdfs_storage
- group_name=group1
- tracker_server=192.168.209.121:22122 -> tracker_server=173.252.200.228:22122
- ##include http.conf ->#include http.conf
接下来会出现很多mkdir data path,这是系统在创建数据目录。
- [2011-10-21 14:49:20] INFO - FastDFS v3.03, base_path=/home/fastdfs/fdfs_storage, store_path_count=1, subdir_count_per_path=256, group_name=group1, run_by_group=, run_by_user=, connect_timeout=30s, network_timeout=60s, port=23000, bind_addr=, client_bind=1, max_connections=256, work_threads=4, disk_rw_separated=1, disk_reader_threads=1, disk_writer_threads=1, disk_rw_direct=0, buff_size=256KB, heart_beat_interval=30s, stat_report_interval=60s, tracker_server_count=1, sync_wait_msec=50ms, sync_interval=0ms, sync_start_time=00:00, sync_end_time=23:59, write_mark_file_freq=500, allow_ip_count=-1, file_distribute_path_mode=0, file_distribute_rotate_count=100, fsync_after_written_bytes=0, sync_log_buff_interval=10s, sync_binlog_buff_interval=10s, sync_stat_file_interval=300s, thread_stack_size=512 KB, upload_priority=10, if_alias_prefix=, check_file_duplicate=0, FDHT group count=0, FDHT server count=0, FDHT key_namespace=, FDHT keep_alive=0, HTTP server port=8888, domain name=
- [2011-10-21 14:49:20] INFO - HTTP supported: server_port=8888, http_trunk_size=262144, default_content_type=application/octet-stream, anti_steal_token=0, token_ttl=0s, anti_steal_secret_key length=0, token_check_fail content_type=, token_check_fail buff length=0
- [2011-10-21 14:49:42] INFO - file: storage_param_getter.c, line: 97, storage_ip_changed_auto_adjust=1, store_path=0, reserved_storage_space=1024 MB, use_trunk_file=0, slot_min_size=0, slot_max_size=16 MB, trunk_file_size=64 MB
- [2011-10-21 14:49:44] INFO - file: tracker_client_thread.c, line: 304, successfully connect to tracker server 173.252.200.228:22122, as a tracker client, my ip is 173.252.200.228
- [2011-10-21 14:50:14] INFO - file: tracker_client_thread.c, line: 1100, tracker server 173.252.200.228:22122, set tracker leader: 173.252.200.228:22122
三:测试及使用FastDFS
1、FastDFS之配置client:
- vi /etc/fdfs/client.conf
- base_path=/home/yuqing/fastdfs-> base_path=/home/fastdfs
- tracker_server=192.168.209.121:22122 -> tracker_server=173.252.200.228:22122
- http.tracker_server_port=8080 ->http.tracker_server_port=9090
- ##include http.conf ->#include http.conf
2、测试上传文件:
- cd ~
- fdfs_test /etc/fdfs/client.conf upload FastDFS_v3.03.tar.gz
执行成功后会出现如下提示:
- This is FastDFS client test program v3.03
- Copyright (C) 2008, Happy Fish / YuQing
- FastDFS may be copied only under the terms of the GNU General
- Public License V3, which may be found in the FastDFS source kit.
- Please visit the FastDFS Home Page http://www.csource.org/
- for more detail.
- [2011-10-21 15:14:59] INFO - base_path=/home/fastdfs, connect_timeout=30, network_timeout=60, tracker_server_count=1, anti_steal_token=0, anti_steal_secret_key length=0
- tracker_query_storage_store_list_without_group:
- server 1. group_name=group1, ip_addr=173.252.200.228, port=23000
- group_name=group1, ip_addr=173.252.200.228, port=23000
- storage_upload_by_filename
- group_name=group1, remote_filename=M00/00/00/rfzI5E6hG_O5wNKxAAVAc72y1Jc.tar.gz
- source ip address: 173.252.200.228
- file timestamp=2011-10-21 15:14:59
- file size=344179
- file crc32=3182613655
- file url: http://173.252.200.228:9090/group1/M00/00/00/rfzI5E6hG_O5wNKxAAVAc72y1Jc.tar.gz
- storage_upload_slave_by_filename
- group_name=group1, remote_filename=M00/00/00/rfzI5E6hG_O5wNKxAAVAc72y1Jc_big.tar.gz
- source ip address: 173.252.200.228
- file timestamp=2011-10-21 15:14:59
- file size=344179
- file crc32=3182613655
- file url: http://173.252.200.228:9090/group1/M00/00/00/rfzI5E6hG_O5wNKxAAVAc72y1Jc_big.tar.gz
3、测试下载文件
在浏览器中,输入上图中的url地址,tracker server会自动重定向到存储文件的storage server,文件下载成功。至此,已经成功搭建了FastDFS,编写你自己的client来进行访问吧!