Background Synthesizing dance motions to match musical inputs is a significant challenge in animation research.Compared to functional human motions,such as locomotion,dance motions are creative and artistic,often infl...Background Synthesizing dance motions to match musical inputs is a significant challenge in animation research.Compared to functional human motions,such as locomotion,dance motions are creative and artistic,often influenced by music,and can be independent body language expressions.Dance choreography requires motion content to follow a general dance genre,whereas dance performances under musical influence are infused with diverse impromptu motion styles.Considering the high expressiveness and variations in space and time,providing accessible and effective user control for tuning dance motion styles remains an open problem.Methods In this study,we present a hierarchical framework that decouples the dance synthesis task into independent modules.We use a high-level choreography module built as a Transformer-based sequence model to predict the long-term structure of a dance genre and a low-level realization module that implements dance stylization and synchronization to match the musical input or user preferences.This novel framework allows the individual modules to be trained separately.Because of the decoupling,dance composition can fully utilize existing high-quality dance datasets that do not have musical accompaniments,and the dance implementation can conveniently incorporate user controls and edit motions through a decoder network.Each module is replaceable at runtime,which adds flexibility to the synthesis of dance sequences.Results Synthesized results demonstrate that our framework generates high-quality diverse dance motions that are well adapted to varying musical conditions and user controls.展开更多
We propose a novel technique to extract features from a range image and use them to produce a 3D pen-and-ink style portrait similar to a traditional artistic drawing. Unlike most previous template-based, component-bas...We propose a novel technique to extract features from a range image and use them to produce a 3D pen-and-ink style portrait similar to a traditional artistic drawing. Unlike most previous template-based, component-based or example-based face sketching methods, which work from a frontal photograph as input, our system uses a range image as input. Our method runs in real-time for models of moderate complexity, allowing the pose and drawing style to be modified interactively. Portrait drawing in our system makes use of occluding contours and suggestive contours as the most important shape cues. However, current 3D feature line detection methods require a smooth mesh and cannot be reliably applied directly to noisy range images. We thus present an improved silhouette line detection algorithm. Feature edges related to the significant parts of a face are extracted from the range image, connected, and smoothed, allowing us to construct chains of line paths which can then be rendered as desired. We also incorporate various portrait-drawing principles to provide several simple yet effective non- photorealistic portrait renderers such as a pen-and-ink shader, a hatch shader and a sketch shader. These are able to generate various life-like impressions in different styles from a user-chosen viewpoint. To obtain satisfactory results, we refine rendered output by smoothing changes in line thickness and opacity. We are careful to provide appropriate visual cues to enhance the viewer＇s comprehension of the human face. Our experimental results demonstrate the robustness and effectiveness of our approach, and further suggest that our approach can be extended to other 3D geometric objects.展开更多
基金Supported by Startup Fund 20019495,McMaster University。
文摘Background Synthesizing dance motions to match musical inputs is a significant challenge in animation research.Compared to functional human motions,such as locomotion,dance motions are creative and artistic,often influenced by music,and can be independent body language expressions.Dance choreography requires motion content to follow a general dance genre,whereas dance performances under musical influence are infused with diverse impromptu motion styles.Considering the high expressiveness and variations in space and time,providing accessible and effective user control for tuning dance motion styles remains an open problem.Methods In this study,we present a hierarchical framework that decouples the dance synthesis task into independent modules.We use a high-level choreography module built as a Transformer-based sequence model to predict the long-term structure of a dance genre and a low-level realization module that implements dance stylization and synchronization to match the musical input or user preferences.This novel framework allows the individual modules to be trained separately.Because of the decoupling,dance composition can fully utilize existing high-quality dance datasets that do not have musical accompaniments,and the dance implementation can conveniently incorporate user controls and edit motions through a decoder network.Each module is replaceable at runtime,which adds flexibility to the synthesis of dance sequences.Results Synthesized results demonstrate that our framework generates high-quality diverse dance motions that are well adapted to varying musical conditions and user controls.
基金Supported by the National Basic Research Program of China (Grant No.2006CB303102)the National Natural Science Foundation of China (Grant Nos.60473103 and 60703028)
文摘We propose a novel technique to extract features from a range image and use them to produce a 3D pen-and-ink style portrait similar to a traditional artistic drawing. Unlike most previous template-based, component-based or example-based face sketching methods, which work from a frontal photograph as input, our system uses a range image as input. Our method runs in real-time for models of moderate complexity, allowing the pose and drawing style to be modified interactively. Portrait drawing in our system makes use of occluding contours and suggestive contours as the most important shape cues. However, current 3D feature line detection methods require a smooth mesh and cannot be reliably applied directly to noisy range images. We thus present an improved silhouette line detection algorithm. Feature edges related to the significant parts of a face are extracted from the range image, connected, and smoothed, allowing us to construct chains of line paths which can then be rendered as desired. We also incorporate various portrait-drawing principles to provide several simple yet effective non- photorealistic portrait renderers such as a pen-and-ink shader, a hatch shader and a sketch shader. These are able to generate various life-like impressions in different styles from a user-chosen viewpoint. To obtain satisfactory results, we refine rendered output by smoothing changes in line thickness and opacity. We are careful to provide appropriate visual cues to enhance the viewer＇s comprehension of the human face. Our experimental results demonstrate the robustness and effectiveness of our approach, and further suggest that our approach can be extended to other 3D geometric objects.
