CANN/runtime模型流切换示例

2_model_switch

【免费下载链接】runtime本项目提供CANN运行时组件和维测功能组件。项目地址: https://gitcode.com/cann/runtime

Description

This sample demonstrates how to use aclmdlRIBuildBegin interface to create a model instance, and implements Stream jumping and Stream activation in tasks.

Product Support

Key interfaces in this sample have the following support status on different products:

Interface	Atlas A3 training series products/Atlas A3 inference series products	Atlas A2 training series products/Atlas A2 inference series products
aclmdlRIBuildBegin	Yes	Yes
aclmdlRIBindStream	Yes	Yes
aclmdlRIEndTask	Yes	Yes
aclmdlRIBuildEnd	Yes	Yes
aclmdlRIUnbindStream	Yes	Yes
aclmdlRIExecuteAsync	Yes	Yes
aclrtSwitchStream	No	Yes
aclrtActiveStream	No	Yes

Build and Run

For environment installation details and running details, see README in the example directory.

Run steps:

# Replace ${install_root} with CANN installation root directory, default installation at /usr/local/Ascend source ${install_root}/cann/set_env.sh export ASCEND_INSTALL_PATH=${install_root}/cann # Replace ${ascend_name} with Ascend AI processor model, obtained by checking Name field using npu-smi info and removing spaces, for example ascend910b3 export SOC_VERSION=${ascend_name} # Some samples involve calling AscendC operators, need to configure AscendC compiler ascendc.cmake path, for example ${install_root}/cann/aarch64-linux/tikcpp/ascendc_kernel_cmake # Find ascendc_kernel_cmake under CANN package installation path, for example find ./ -name ascendc_kernel_cmake, and replace ${cmake_path} with ascendc_kernel_cmake directory path export ASCENDC_CMAKE_DIR=${cmake_path} # Build and run bash run.sh

CANN RUNTIME API

Key features and interfaces in this sample:

• Initialization
  • Call aclInit interface to initialize AscendCL configuration.
  • Call aclFinalize interface to deinitialize AscendCL.
• Device Management
  • Call aclrtSetDevice interface to specify Device for computation.
  • Call aclrtResetDeviceForce interface to forcibly reset current computation Device and reclaim Device resources.
• Context Management
  • Call aclrtCreateContext interface to create Context.
  • Call aclrtDestroyContext interface to destroy Context.
• Stream Management
  • Call aclrtCreateStream interface to create Stream.
  • Call aclrtSynchronizeStream interface to block waiting for Stream task completion.
  • Call aclrtDestroyStreamForce interface to forcibly destroy Stream, discarding all tasks.
  • Call aclrtCreateStreamWithConfig interface to create Stream with special config.
  • Call aclrtSwitchStream interface to jump between Streams based on condition.
  • Call aclrtActiveStream interface to activate Stream.
• Model Management
  • Call aclmdlRIBuildBegin interface to start building a model runtime instance.
  • Call aclmdlRIBindStream interface to bind model runtime instance to Stream.
  • Call aclmdlRIEndTask interface to mark task dispatch end.
  • Call aclmdlRIBuildEnd interface to finish building model runtime instance.
  • Call aclmdlRIUnbindStream interface to unbind model runtime instance from Stream.
  • Call aclmdlRIExecuteAsync interface to asynchronously execute model inference.
  • Call aclmdlRIDestroy interface to destroy model runtime instance.
• Memory Management
  • Call aclrtMalloc interface to allocate Device memory.
  • Call aclrtFree interface to release Device memory.
• Data Transfer
  • Call aclrtMemcpy interface to implement data transfer by memory copy.
  • Call aclrtMemcpyAsync interface to perform asynchronous memory copy.

Known Issues

None

【免费下载链接】runtime本项目提供CANN运行时组件和维测功能组件。项目地址: https://gitcode.com/cann/runtime