The smart grid has been revolutionizing electrical generation and consumption through a two-way flow of power and information. As an important information source from the demand side, Advanced Metering Infrastructure ...The smart grid has been revolutionizing electrical generation and consumption through a two-way flow of power and information. As an important information source from the demand side, Advanced Metering Infrastructure （AMI） has gained increasing popularity all over the world. By making full use of the data gathered by AMI, stakeholders of the electrical industry can have a better understanding of electrical consumption behavior. This is a significant strategy to improve operation efficiency and enhance power grid reliability. To implement this strategy, researchers have explored many data mining techniques for load profiling. This paper performs a state-of-the-art, comprehensive review of these data mining techniques from the perspectives of different technical approaches including direct clustering, indirect clustering, clustering evaluation criteria, and customer segmentation. On this basis, the prospects for implementing load profiling to demand response applications, price-based and incentivebased, are further summarized. Finally, challenges and opportunities of load profiling techniques in future power industry, especially in a demand response world, are discussed.展开更多
As social media and online activity continue to pervade all age groups, it serves as a crucial platform for sharing personal experiences and opinions as well as information about attitudes and preferences for certain ...As social media and online activity continue to pervade all age groups, it serves as a crucial platform for sharing personal experiences and opinions as well as information about attitudes and preferences for certain interests or purchases. This generates a wealth of behavioral data, which, while invaluable to businesses, researchers, policymakers, and the cybersecurity sector, presents significant challenges due to its unstructured nature. Existing tools for analyzing this data often lack the capability to effectively retrieve and process it comprehensively. This paper addresses the need for an advanced analytical tool that ethically and legally collects and analyzes social media data and online activity logs, constructing detailed and structured user profiles. It reviews current solutions, highlights their limitations, and introduces a new approach, the Advanced Social Analyzer (ASAN), that bridges these gaps. The proposed solutions technical aspects, implementation, and evaluation are discussed, with results compared to existing methodologies. The paper concludes by suggesting future research directions to further enhance the utility and effectiveness of social media data analysis.展开更多
There is a need to obtain the hydrologic data including ocean current, wave, temperature and so on in the South China Sea. A new profiling instrument which does not suffer from the damage due to nature forces or incid...There is a need to obtain the hydrologic data including ocean current, wave, temperature and so on in the South China Sea. A new profiling instrument which does not suffer from the damage due to nature forces or incidents caused by passing ships, is under development to acquire data from this area. This device is based on a taut single point mid-water mooring system. It incorporates a small, instrumented vertically profiling float attached via an electromechanical cable to a winch integral with the main subsurface flotation. On a pre-set schedule, the instrument float with sensors is winched up to the surface if there is no ship passing by, which is defined by an on-board miniature sonar. And it can be. inunediately winched down to a certain depth if the sonar sensor finds something is coming. Since, because of logistics, the area can only be visited once for a long time and a minimum of 10 times per day profiles are desired, energy demands are severe. To respond to these concerns, the system has been designed to conserve a substantial portion of the potential energy lost during the ascent phase of each profile and subsequently use this energy to pull the instrument down. Compared with the previous single-point layered measuring mode, it is advanced and economical. At last the paper introduces the test in the South China Sea.展开更多
The objective of this paper is to propose an adjustment to the three methods of calculating the probability that regularities in a sample data represent a systemic influence in the population data. The method proposed...The objective of this paper is to propose an adjustment to the three methods of calculating the probability that regularities in a sample data represent a systemic influence in the population data. The method proposed is called data profiling. It consists of calculating vertical and horizontal correlation coefficients in a sample data. The two correlation coefficients indicate the internal dynamic or inter dependency among observation points, and thus add new information. This information is incorporated in the already established methods and the consequence of this integration is that one can conclude with certainty that the probability calculated is indeed a valid indication of systemic influence in the population data.展开更多
Data governance is a subject that is becoming increasingly important in business and government. In fact, good governance data allows improved interactions between employees of one or more organizations. Data quality ...Data governance is a subject that is becoming increasingly important in business and government. In fact, good governance data allows improved interactions between employees of one or more organizations. Data quality represents a great challenge because the cost of non-quality can be very high. Therefore the use of data quality becomes an absolute necessity within an organization. To improve the data quality in a Big-Data source, our purpose, in this paper, is to add semantics to data and help user to recognize the Big-Data schema. The originality of this approach lies in the semantic aspect it offers. It detects issues in data and proposes a data schema by applying a semantic data profiling.展开更多
基金supported by the National Science Fund for Distinguished Young Scholars (No. 51325702)
文摘The smart grid has been revolutionizing electrical generation and consumption through a two-way flow of power and information. As an important information source from the demand side, Advanced Metering Infrastructure （AMI） has gained increasing popularity all over the world. By making full use of the data gathered by AMI, stakeholders of the electrical industry can have a better understanding of electrical consumption behavior. This is a significant strategy to improve operation efficiency and enhance power grid reliability. To implement this strategy, researchers have explored many data mining techniques for load profiling. This paper performs a state-of-the-art, comprehensive review of these data mining techniques from the perspectives of different technical approaches including direct clustering, indirect clustering, clustering evaluation criteria, and customer segmentation. On this basis, the prospects for implementing load profiling to demand response applications, price-based and incentivebased, are further summarized. Finally, challenges and opportunities of load profiling techniques in future power industry, especially in a demand response world, are discussed.
文摘As social media and online activity continue to pervade all age groups, it serves as a crucial platform for sharing personal experiences and opinions as well as information about attitudes and preferences for certain interests or purchases. This generates a wealth of behavioral data, which, while invaluable to businesses, researchers, policymakers, and the cybersecurity sector, presents significant challenges due to its unstructured nature. Existing tools for analyzing this data often lack the capability to effectively retrieve and process it comprehensively. This paper addresses the need for an advanced analytical tool that ethically and legally collects and analyzes social media data and online activity logs, constructing detailed and structured user profiles. It reviews current solutions, highlights their limitations, and introduces a new approach, the Advanced Social Analyzer (ASAN), that bridges these gaps. The proposed solutions technical aspects, implementation, and evaluation are discussed, with results compared to existing methodologies. The paper concludes by suggesting future research directions to further enhance the utility and effectiveness of social media data analysis.
基金The project was financially supported by the High Tech Research and Development (863) Program (Grant No2005AA604220)by a grant from China National Offshore Oil Corporation (Grant No051100036)
文摘There is a need to obtain the hydrologic data including ocean current, wave, temperature and so on in the South China Sea. A new profiling instrument which does not suffer from the damage due to nature forces or incidents caused by passing ships, is under development to acquire data from this area. This device is based on a taut single point mid-water mooring system. It incorporates a small, instrumented vertically profiling float attached via an electromechanical cable to a winch integral with the main subsurface flotation. On a pre-set schedule, the instrument float with sensors is winched up to the surface if there is no ship passing by, which is defined by an on-board miniature sonar. And it can be. inunediately winched down to a certain depth if the sonar sensor finds something is coming. Since, because of logistics, the area can only be visited once for a long time and a minimum of 10 times per day profiles are desired, energy demands are severe. To respond to these concerns, the system has been designed to conserve a substantial portion of the potential energy lost during the ascent phase of each profile and subsequently use this energy to pull the instrument down. Compared with the previous single-point layered measuring mode, it is advanced and economical. At last the paper introduces the test in the South China Sea.
文摘The objective of this paper is to propose an adjustment to the three methods of calculating the probability that regularities in a sample data represent a systemic influence in the population data. The method proposed is called data profiling. It consists of calculating vertical and horizontal correlation coefficients in a sample data. The two correlation coefficients indicate the internal dynamic or inter dependency among observation points, and thus add new information. This information is incorporated in the already established methods and the consequence of this integration is that one can conclude with certainty that the probability calculated is indeed a valid indication of systemic influence in the population data.
文摘Data governance is a subject that is becoming increasingly important in business and government. In fact, good governance data allows improved interactions between employees of one or more organizations. Data quality represents a great challenge because the cost of non-quality can be very high. Therefore the use of data quality becomes an absolute necessity within an organization. To improve the data quality in a Big-Data source, our purpose, in this paper, is to add semantics to data and help user to recognize the Big-Data schema. The originality of this approach lies in the semantic aspect it offers. It detects issues in data and proposes a data schema by applying a semantic data profiling.
