In recent years, with the rapid development of data intensive applications, data replication has become an enabling technology for the data grid to improve data availability, and reduce file transfer time and bandwidt...In recent years, with the rapid development of data intensive applications, data replication has become an enabling technology for the data grid to improve data availability, and reduce file transfer time and bandwidth consumption. The placement of replicas has been proven to be the most difficult problem that must be solved to realize the process of data replication. This paper addresses the quality of service （QoS） aware replica placement problem in data grid, and proposes a dynamic programming based replica placement algorithm that not only has a QoS requirement guarantee, but also can minimize the overall replication cost, including storage cost and communication cost. By simulation, experiments show that the replica placement algorithm outperforms an existing popular replica placement technique in data grid.展开更多
MapReduce is a popular program- ming model for processing large-scale datasets in a distributed environment and is a funda- mental component of current cloud comput- ing and big data applications. In this paper, a hea...MapReduce is a popular program- ming model for processing large-scale datasets in a distributed environment and is a funda- mental component of current cloud comput- ing and big data applications. In this paper, a heartbeat mechanism for MapReduce Task Scheduler using Dynamic Calibration （HMTS- DC） is proposed to address the unbalanced node computation capacity problem in a het- erogeneous MapReduce environment. HMTS- DC uses two mechanisms to dynamically adapt and balance tasks assigned to each com- pute node： 1） using heartbeat to dynamically estimate the capacity of the compute nodes, and 2） using data locality of replicated data blocks to reduce data transfer between nodes. With the first mechanism, based on the heart- beats received during the early state of the job, the task scheduler can dynamically estimate the computational capacity of each node. Us- ing the second mechanism, unprocessed Tasks local to each compute node are reassigned and reserved to allow nodes with greater capacities to reserve more local tasks than their weaker counterparts. Experimental results show that HMTS-DC performs better than Hadoop and Dynamic Data Placement Strategy （DDP） in a dynamic environment. Furthermore, an en- hanced HMTS-DC （EHMTS-DC） is proposed bv incorporatin historical data. In contrastto the ＂slow start＂ property of HMTS-DC, EHMTS-DC relies on the historical computation capacity of the slave machines. The experimental results show that EHMTS-DC outperforms HMTS-DC in a dynamic environment.展开更多
CORBA (Common Object Request Broker Architecture) provides 16 Common Object Services for distributed application development, but none of them are fault tolerance related services. In this paper, we propose a replica...CORBA (Common Object Request Broker Architecture) provides 16 Common Object Services for distributed application development, but none of them are fault tolerance related services. In this paper, we propose a replicated object based Fault Tolerant Object Service (FTOS) for CORBA environment. Two fault tolerant mechanisms are provided in FTOS including dynamic voting mechanism and object replication mechanism. The dynamic voting mechanism uses majority voting strategy to ensure object state consistency in failure situations. The object replication mechanism can help system administrators to replicate and start up objects easily. Our implementation provides a library according to the style of COSS. With this library, programmers can develop distributed applications with fault tolerance capability very easily.展开更多
基金sponsored by the National Natural Science Foundation of China (61202354)the Hi-Tech Research and Development Program of China (2007AA01Z404)Scientific & Technological Support Project (Industry) of Jiangsu Province (BE2011189)
文摘In recent years, with the rapid development of data intensive applications, data replication has become an enabling technology for the data grid to improve data availability, and reduce file transfer time and bandwidth consumption. The placement of replicas has been proven to be the most difficult problem that must be solved to realize the process of data replication. This paper addresses the quality of service （QoS） aware replica placement problem in data grid, and proposes a dynamic programming based replica placement algorithm that not only has a QoS requirement guarantee, but also can minimize the overall replication cost, including storage cost and communication cost. By simulation, experiments show that the replica placement algorithm outperforms an existing popular replica placement technique in data grid.
文摘MapReduce is a popular program- ming model for processing large-scale datasets in a distributed environment and is a funda- mental component of current cloud comput- ing and big data applications. In this paper, a heartbeat mechanism for MapReduce Task Scheduler using Dynamic Calibration （HMTS- DC） is proposed to address the unbalanced node computation capacity problem in a het- erogeneous MapReduce environment. HMTS- DC uses two mechanisms to dynamically adapt and balance tasks assigned to each com- pute node： 1） using heartbeat to dynamically estimate the capacity of the compute nodes, and 2） using data locality of replicated data blocks to reduce data transfer between nodes. With the first mechanism, based on the heart- beats received during the early state of the job, the task scheduler can dynamically estimate the computational capacity of each node. Us- ing the second mechanism, unprocessed Tasks local to each compute node are reassigned and reserved to allow nodes with greater capacities to reserve more local tasks than their weaker counterparts. Experimental results show that HMTS-DC performs better than Hadoop and Dynamic Data Placement Strategy （DDP） in a dynamic environment. Furthermore, an en- hanced HMTS-DC （EHMTS-DC） is proposed bv incorporatin historical data. In contrastto the ＂slow start＂ property of HMTS-DC, EHMTS-DC relies on the historical computation capacity of the slave machines. The experimental results show that EHMTS-DC outperforms HMTS-DC in a dynamic environment.
文摘CORBA (Common Object Request Broker Architecture) provides 16 Common Object Services for distributed application development, but none of them are fault tolerance related services. In this paper, we propose a replicated object based Fault Tolerant Object Service (FTOS) for CORBA environment. Two fault tolerant mechanisms are provided in FTOS including dynamic voting mechanism and object replication mechanism. The dynamic voting mechanism uses majority voting strategy to ensure object state consistency in failure situations. The object replication mechanism can help system administrators to replicate and start up objects easily. Our implementation provides a library according to the style of COSS. With this library, programmers can develop distributed applications with fault tolerance capability very easily.
