guys who hav calculated the number of CPU cycles needed to set up a monitor point ?

CPU cycles? Why? I mean, it’s a pointless metric

mmmm well probably in my case i should consider this metric because i want to monitor in a "synchrone way" the page table entries of a virtual address , for example lets suppose that to Monitor an entry this will take a long time so everything will be non consistent if the os mean while changed the config entries while setting the monitor

im not sure about this just im trying to be open mind to any problem of synchro that may happen

I see your point. Those are the implementation details which I'm not aware of then. HyperDbg aside, whenever you have a switch to a VMX root mode (where the hypervisor lives), VMX guest mode (with the OS) is not running. Think of the same CPU just switching between different modes. But again, it should be implementation specific and I'm not an expert in HyperDbg

What I do in my debugger is "break-in" first - i.e. freeze vmx guest execution on all cores, and then set up breakpoints etc. This way the OS is not running and I may expect what you call a synchronized execution.

this idea is great to freez execution on all cores while setting up the breakpoints i will have a look into the implementation of the break in first , to add it to my project

actually im turning hyperdbg from "debugger project" to " EDR/seucrity monitor solution " xd

Couldn't you fix it by using spinlock functions? 🤔

https://docs.hyperdbg.org/commands/scripting-language/functions/spinlocks

For HyperDbg, it's the same for breakpoints. Once we want to halt the entire execution, one of the cores triggers NMIs in all of the cores, and NMIs will cause a VM-exit on all cores. Then we spin on a core-specific spinlock so all of the cores are halted and the user could apply the breakpoint.

uint32

I will look for this solution too ... meanwhile I will assume that no synchro problem will occur and let's see what will happen ... page tables are a real headache xd

Btw sometimes when I try to remove a single hook by calling "ConfigureEptHookUnhookSingleAddress , it doesn't succeed in the first occurrence , i have to call it in a while loop what can be the root cause of this behavior, a synchrone problem between cores ?

Yeah, agree. There are also a couple of interlocked (exchange) atomic functions, you might also use them for fixing some synchronization scenarios:

https://docs.hyperdbg.org/commands/scripting-language/functions/interlocked

Well, it should generally remove the hook on the first try unless there is a bug. One possible scenario occurs when the target generates a high rate of events, and at some point, an event is triggered on a neighboring core while you simultaneously remove the event on another core. As a result, the event may still occur even after it has been deleted.

This scenario won't happen in HyperDbg's debugger mode because all cores are halted using an NMI (when not in VMX-root mode). If a core is caught with an NMI in VMX-root mode, HyperDbg allows it to complete the VM-exit before triggering a second VM-exit to halt the core. So, you're basically sure that the neighboring cores aren't triggering an event while you're removing the hook.

mmmmmmmmm i have to investigate this issue , what startegy i should follow to keep track of where the unhooked process was stopped

basically i will not work with "event" concept in my malware edr solution xd i will try to use the memory monitoring features of hyperdbg as an API

You can check for this flag on all vCPUs. It's not 100% accurate as you might catch the flag (before or after) this flag is set to TRUE/FALSE but should be enough for a test.

okaaaaaay

got it it got it

yes it may help as in indication of the current mode

When will remote debugging via ethernet be available?

Probably not soon. It's so hard to implement communication for HyperDbg over Ethernet since it needs to support different PCIe devices like Intel e1000 NIC and other NIC vendor like Realtek. And the only reference that I know for it, is the Linux kernel.

In Windows SDK we can find the implementation of RealTek driver for KDnet which is fine but they also have the implementation of Intel NIC but for some unknown reasons, their solution doesn't have some of the header files and it's not compilable.

If you guys know a clean (preferably OS-independent) implementation of Intel e1000 or other NIC vendors that can be compiled and used as a module that is also not dependent to certain OS functions, pls send it to me.

Other than that, another option is having the transportation layer over USB. From what I know, USB is not exported as a PCIe interface (even though xHCI seems to be a part of PCIe) but I'm not sure how to communicate with USB and how USB relates to xHCI over PCIe. And I don't know if the implementation of USB is vendor-specific (like different NICs) or not?

Maybe @honorary_bot knows the answer to these questions. 🤔

Its not, because there is a host controller interface standard for that. You probably don't want USB2 (ehci debug port extension), because there are no EHCI controllers around and no devices supporting that anymore. But USB3+ which has XHCI controller have USB DbC feature for debugging, though HCI revision should be >0.99 or Skylake+ PCH.

I mean, both XHCI and EHCI are PCI devices of some sort

So the communication is over normal PCIe BARs?

Simply put - yes

e1000 is relatively easy to implement btw, but I'm not sure about environment independent version. Usually you have at least some kind of execution environment with runtime.

The problem is that in VMX-root mode, we couldn't rely on any Windows-specific functions since paging is disabled (RFLAGs'IF). So, that's why a platform-independent implementation is needed.

Realtek is weird, in my case it randomly works for seemungly the same devices

So, do you think the easiest option for us (which we don't need to go through different vendors) is xHCI?

What I mean is if I were to share my driver, it would still utilize a lot of runtime functions and different subsystems. You would need to adapt your project for that way too much.

No, rather e1000

XHCI has some bugs, working them around will take time

There is no defined role detection mechanism when DbC enabled, that's why it's startup is not reliable

and other than that, I've heard USB is a pooling mode device. Is it true? In that case if it's over PCIe then we don't have MSI/MSI-X?

I had to use Beagle device to try and fix that and still it's not 100% success rate.

This big boy https://evision-webshop.eu/total-phase/beagle-5000-v2-usb-3.0-ultimate-protocol-analyzer?gad_source=1&gclid=CjwKCAiArKW-BhAzEiwAZhWsIE0-tlYKftKBmYZy8yNPXMKdBaS_HNzxbc0xIM88pVw8Ignnugq0wRoCXtAQAvD_BwE

I don't use interrupts in my hypervisor at all, my drivers use polling mode.

XHCI DbC feature is cool, because you can share your XHCI device with the OS

XHCI marks your port busy

In the case with NIC, I just take it over and hide from the OS, because you can't share it

BTW, even if the device supports SR-IOV, you can't just steal it and virtualize. PF and VF functions look different and have different drivers, so the OS will not be happy with that

The problem with e1000 is that we could only use over virtual devices (e.g., VMware's nested virtualization or KVM's nVMX). I'm not sure if regular Intel NICs also have a legacy e1000 mode or not.

Let's put it this way, all my NUCs from Sandy Bridge to RPL platforms work :)

*with the internal intel NIC

Work without driver modification

By SR-IOV you mean if the NIC have different functions (as functions in BDF?)?

I only developed it once and never adjusted

Yeah, not just nics, but in general. SR-IOV is not a silver bullet for our use case.

I don't understand. Is it like a legacy feature in all Intel NICs?

Because as far as I know, e1000 is not a real NIC, it's just Intel's standard way of communication for NICs.

It's not a feature or something. You have a family of NICs that have the same programming interface.

Exactly

Well, it is a set of real NICs that are programmed with this interface

Ah, okay got it. So, basically most of Intel's NICs also support the same interface similar to e1000.

This is really good 👌

If a NIC has a super new cool feature, it is programmed in its PHY, so that you need a special driver. But if you don't care and just want to send and receive packets - e1000 is the answer.

Most client desktop NICs

Because there are NICs like 800 series, 700 series, I350 etc - those are more server ones (and expensive). They have a different programming interface, though it is also documented.

Whenever I didn't have a working NIC, I would use a discrete one - Intel Gigabit CT Desktop Adapter

It works and it used to be relatively cheap

That's great.

Anyway, thank you @honorary_bot very much for all the valuable information. 🙏

My pleasure!

I now dream of implementing a USB4 \ Thunderbolt support, but its... It's not gonna be easy.

The worst one for me was the FireWire

Not recommended for driver development :)

Thunderbolt? Isn't it like a normal PCIe BAR driver development? Does it need anything special other than using the same PCIe drivers?

Well BARs are just a placeholders for the MMIO interface of the device, right? Every device or host controller has it's own interface and programming stack. And if e1000 is just a bunch of MMIO regs with descriptors that would point to your ethernet frame data, then for FireWire for example you need to also implement a 1394 stack to be useful.

For USB4 you would need to implement a whole bunch of protocols and entities, it's kinda complicated

oh got it.

👍

[discord] <btylor0> Is there any support for debugging a VM with nested virtualization. The use case being having a HyperDbg cloud instance or proxmox server KVM instance running host windows, running vmware target

HyperDbg is developed and well tested on VMware's nested virtualization (Workstation and ESXi), and physical machines. Other nested virtualization environments either not work or will crash at some points. But, it's on the todo list to support them in the future.

[discord] <btylor0> Using vmware workstation pro 17.5.x+ there is an option for BIOS or UEFI, for EfiGuard to work I need to use UEFI but it seems it keeps hitting a driver verifier dma violation (just running the VM for the first time before EfiGuard is downloaded). It also only happens when the IOMMU processor setting is enabled, which works fine on the BIOS mode. Has anybody had this issue?

[discord] <btylor0> Windows 11 worked, but 10 doesn't seem to with UEFI mode. Now I'm having an issue with connecting. EfiGuard is loaded on the VM and says its loaded correctly and that DSE is disabled, when I connect it gives:

err, OpenSCManager failed (5)
unable to install VMM driver
err, OpenSCManager failed (5)
failed to install or load the driver
Fix for OpenSCManager was to change hyperdbg-cli properties to run as admin

Not sure if I understand what you mean. But I don't know if emulating IOMMU in VMware Workstation does anything special or what its functionality is since they don't support PCIe passthrough in VMware Workstation. I could imagine what it could do on an ESXi machine but not in VMware Workstation. They also don't reveal VT-d support (emulation) to the guest. 🤔

So, the problem is that you use EfiGuard on VMware workstation and it works on HyperDbg running in Windows 11 but not in Windows 10? Am I get it correctly?

[discord] <btylor0> That was the issue, I would just get BSODS with the virtualization and UEFI mode in the VM settings, it wouldn't boot. I'm not sure what the issue was, but win 11 fixed it, everything is new releases, including the ISOs from windows insider preview so that might have been it.

Guys one synchronization problem I'm trying to find the best solution for :

1- let's say are monitoring a PML4 entry , the goal is track any possible changes in the address translation /presence of an equivalent physical address etc.

2- let's say that os decided to paged out the page so it will try to set present bit to zero (PML4E.presentbit ==0) , okay we will have an EPT violation with write access, everything looks fine until now all we have to do is to read the new content of the PML4E in the MFT exit handler (after the execution of the assembly instruction that attempts to make the change) to analyze the change

3- now in the mft handler I will try to unmonitor all related entries since they are not relevant anymore after the paged was paged out (pdpe , pde, pte), the problem is that all I can do is to unhook these entries for a single core since we are in vmx root mode and we cannot synchronize via dpc and for other cores the ept config is the same so any further write access will not reflect reality and it will be a mess , what is the best solution we can think of

[discord] <btylor0> spinlock

You're probably talking about flushing the EPT translation cache. You'd have to broadcast invvpid or invept (depending on the implementation) for all cores using either IPI or some software defined mailbox (synchronizing and checking messages manually).

yes exactly i want o broadcast this event for all cores to flash their EPT translation cache and restore the original configuration however from my understanding we cannot use IPI so im afraid that some miss synchro in that time window

I see, this is hyperdbg implementation specific, summon @HughEverett

Yes, but the problem here is that HyperDbg uses NMIs to halt all cores in VMX-root mode (and VMX non-root) mode. This is find but the problem is other cores might be in the middle of handling a VM-exit which is related to current EPT violation (that we want to remove the hook as a result of MTF). This is not possible in HyperDbg as we will end up removing hooks (internal HyperDbg structures that store the details of each hook). It might not be an issue in most of the HyperDbg events but in events with a high rate of execution, it's problematic.

If you see HyperDbg documentation for removing events using the 'events' command this problem is described.

How about using IPI with INIT? By design it forces cores to vmexit

https://docs.hyperdbg.org/commands/debugging-commands/events#remarks

Unless they were in VMX root of course, but in this case you can synchronize them with a barrier

This problem does not make sense in VMI mode, because basically everything is running so it never happens but in case of Debugger mode, we just disable the event (by event I mean HyperDbg defined events not Intel events) and then clear them later using DPCs.

Do you mean if the core is in VMX root non-root mode?

If a core was in VMX guest, it will vmexit with INIT exit reason, it will not restart the core

Well, our solution is almost similar. We will broadcast NMIs to all cores. (NMIs will cause VM-exit in cores that are not in VMX root-mode) and will cause a HOST_IDT catch of the NMI if the core is in VMX root-mode. The problem here is not related to Intel, it's possible to invalidate all of the cores by broadcasting, the problem is that HyperDbg also needs to keep the track of EPT hooks in its internal structures, that's why we won't broadcast all of the INVEPTs (or INVVPIDs) immediately but in theory it's possible but the way that we solved this problem is we disabled the event and invalidate entries at the next DPC.

I see

I think the solution to this problem is that you disable the event (the Event structure of HyperDbg). It could be done by simply using the same functions that we use for removing events (using the 'events' command). It disables the event for you and later removes it when the state of the system is safe. Meanwhile, EPT violations might happen but HyperDbg won't notify you because you already flagged the event for disabling it.

i got your idea bro (wait until the system in a safe stat then remove the event)

but in my design of my edr im almost working without events concept (defined by hyperdbg for sure) so i think the best thing i can do is to define some sort of pending list of hooks that the handler will check and clear before it start handling a new MFT exit , i know it may cost some performance

but tha my shitty edr design was like that xd

Yes, that's also possible. Usually these logical concepts are handled in hyperkd.sys (which you're not using it) and hyperhv.dll remains as a bare bone for the hypervisor functionalities.

yes you are right i have read some part of the code source of hyperkd and i saw the logic handling of similiar type of scenarios

hyperhv.dll is the best API for hypervisor functionalities for me its like a complete framework to build your own hypervisor specifications

guys , any idea why this function should not be called from VMX root mode , what are the risks exactly? (maybe it will cause some event handling related to cr3??!) UINT64
PhysicalAddressToVirtualAddressByCr3(PVOID PhysicalAddress, CR3_TYPE TargetCr3)
{
CR3_TYPE CurrentProcessCr3;
UINT64 VirtualAddress;
PHYSICAL_ADDRESS PhysicalAddr;

//
// Switch to new process's memory layout
//
CurrentProcessCr3 = SwitchToProcessMemoryLayoutByCr3(TargetCr3);

//
// Validate if process id is valid
//
if (CurrentProcessCr3.Flags == NULL64_ZERO)
{
//
// Pid is invalid
//
return NULL64_ZERO;
}

//
// Read the virtual address based on new cr3
//
PhysicalAddr.QuadPart = (LONGLONG)PhysicalAddress;
VirtualAddress = (UINT64)MmGetVirtualForPhysical(PhysicalAddr);

//
// Restore the original process
//
SwitchToPreviousProcess(CurrentProcessCr3);

return VirtualAddress;
}

It's because the hypervisor (in this case hyperhv.dll) might not be mapped in the target process memory layout. Most of the cases, it won't be problematic but I see random cases where the HyperDbg codes are not mapped into certain CR3s.

Well in my case this will cause me a problem because if i cannot use PhysicalAddresdToVirtualAddressByCr3() (from the view of the kernel) i will not have a mapped virtual address that i will use to read the new content of page table entries to analyze the new change in the MFT handler so i know what i should do (re translate the addresd, unmonitor something etc)

Because in the end the cpu need a virtual address to access the content of a memory, he cannot work with the physical

Or i have to supposed that function will always return me a valid virtual address from the kernel view equivalent to a page table entry physicsl address ?

Well, I'm not sure if I understand what you mean but CPU in the end needs a physical address to access memory (EPT is all about converting Guest PA to host PA).

Are you talking about regular OS page tables or EPT page tables?

🤔

No I'm talking about regular page tables

I'm trying to monitor the os page tables entries using ept

I will give you an example...

Let's say we have a user mode virtual address 0x7ff5729 , in the initial phase this virtual address has 0x1000 as en equivalent physical address

However at some point the OS decided to paged out the page so i should not monitor anymore the physical address 0x1000 and in parallel I should monitor in real time all "os" page tables entries involved in the address translation (pml4e , pdpe, pde, pte)

The problem is that I need to read the content of these entries in VMX-root mode and I have only physical addresses

And the is a problem since the cpu need a virtual address to access the content of a page table entry (aka to deference the pointer)

I don't know if the problem is cleat

Clear* now?

There are plenty of functions in MemoryMapper.c for reading physical memory, in this case you should use this one.

This function gets a physical address as the input and saves the content of memory to the target virtual address.

As a general rule, those functions that have 'Safe' in their function name are safe to be used in VMX-root mode.

I hope they are safe to use in vmx root mode ..

I will test it

For sure

Yes, these 'Safe' functions are all okay to be used in VMX-root. You can also use them simultaneously in different cores (there is no overlap).

Got it i will try to understand how they work internally because I have implemented similar function but it only works on vmx no root mode since it use Mmcopymemory (<= APC_LEVEL)

No, MmCopyMemory does not work on MMIO ranges where its not mapped in the page table. It's always better to use internal HyperDbg functions for reading physical memory.

Other than that, there are a couple of VMCALLs for reading physical memory (from VMX non-root mode), you could use them as well.

Mmmmm okay bro got it I will try to read the source code of these functions and use them in my edr and let's see how things will go ^^

Btw I was reading your blog about pfn stuffs

__int64 __fastcall MmGetVirtualForPhysical(unsigned __int64 a1) { return (a1 & 0xFFF) + (*(_QWORD *)(48 * (a1 >> 12) - 6047313952760i64) << 25 >> 16); }

MmGetVirtualForPhysical how this fucking weird function works 😂😂 I mean it needs now context or what (like cr3 value)?

This is my question too. I know that there is a random number here which refers to the page table's randomized self-ref entry, but I don't know exactly how it works.

mmmm well yes this function drive me crazy i mean i cannot understand how the result of MmGetVirtualForPhysical will be diffirent if we change the cr3 based on this implementation

@HughEverett hey, pls tell once again, how can i fight int1 and popfq calls? i need debugger to ignore them

Do you mean #DB for trap flags?

@HughEverett bro what is the advantage of having a specific eptpagetable for each core

From a design perspective

I mean for a better control,it will be easy that all cores share the same EptPageTable ?

mm not sure, i didnt enable those traps, or they are enabled by default ?

They're enabled by default but you could disable them using:

test trap off

Or if you want HyperDbg not to intercept breakpoints (#BP), you could run:

test breakpoint off

If you use these commands, you could use other debuggers like WinDbg or x64dbg normally as HyperDbg won't handle breakpoints and traps.

HyperDbg was one a single EPTP until 0.7 (I think 🤔), but then we make it a per core multiple EPTP which was a huge milestone for us. Lots of potential bugs are solved with that release.

Well, the advantage is if you just use a single EPTP, you might loose some EPT hooks or Monitor hook specially if there is a high rate of execution. For example, you might manipulate one EPTP entry and another core might simultaneously wants to modify the same row and you could imagine that one of the cores will have a wrong EPT page table entry.

This is not a problem that could be solved by a spinlock, because this way you'll introduce a lot of delays which is a design disaster.

Besides that, you may need to modify the memory layout of a single thread while maintaining different memory layouts for other threads running on separate cores. In all these scenarios, multiple EPTPs are required.

As I mentioned earlier, supporting this was a significant milestone for HyperDbg. Before we transitioned to the current design, I was in contact with some KVM developers, all of them criticized HyperDbg’s approach for using a single EPTP.

Oh got it ... it will be a real mess if two cores tried to access simultaneously the same eptp entry , 100% they will be some sort of inconsistencies information

Now I'm aware that hyperdbg use a eptp table for each core

I have to update my code in mft where I'm catching most important events after write modification handler to force each core to flush its cache and restore the original entry

In my current code I thought that if one core changed the eptp entry it will be sufficient since all cores share the same table

But no no no that's not the case haha

@HughEverett bro from my new understanding , i should now call

BOOLEAN ConfigureEptHookMonitor(UINT32 CoreId, EPT_HOOKS_ADDRESS_DETAILS_FOR_MEMORY_MONITOR * HookingDetails,UINT32 ProcessId)

for all existing cores to make sure that for the address we want to monitor all cores will have the same access policy configuration

right

I'm not sure what you mean by "same access policy configuration" but generally once you request a Monitor hook, it synchronizes itself with all of the cores. So, it means you don't need to apply hooks for each core separately, HyperDbg will synchronize it for you.

And it does this by knowing whether you're applying the hook in the debugger mode or the VMI mode, because synchronization of each of them is different.

Well you get my point exactly

I'm worried about synchronizing the hook requests for all cores

But actually I'm not working with debugger mode 😕 or the vmi mode either, I told last time that I'm building a simple edr based on hyperdbg API(s)

So it looks like I should handle this task of synchronizing the memory layout for all cores ... I know that when you install a hook from vmx no root mode hyperdbg will make a generic broadcast to apply the same hook for all cores

But in my case most I have to make some ept config change in the MFT handler so i should handle it by myself since there is no DPC mechanism

If I'm not wrong

Well, I'm not sure if I implement the broadcasting mechanism in hyperkd or hyperhv but I will look into it for you later once I have access to my laptop. But generally if it's broadcasted from hyperkd, you can copy the codes from broadcaster to your project and use it the same way as hyperkd.sys uses it.

So, it shouldn't be a big problem, just take a look at I think in 'Debugger.c' in hyperkd projects there should be some reference to functions that apply event, you could call the same function and it will be fine.

There should be a flag that you pass to the top level function and you'll specify whether you want to apply it from VMX root-mode (which is for the debugger mode) or apply it from VMX non-root mode (which is for your case in the vmi mode) and that's all.

I got your point bro thanks for help

Its not easy to build a new project based on a big project like hyeperdbg😂 it's literally means u have to read and understand the design from scratch of the whole project

👍

Bro one last question .. i want now to test my edr based on hyperdbg , how to take of the advantage of LogInfo() etc. It's bit weird here for me singe I'm not using hyperdbg as debugger xd

I mean specially my code that will take of hyperdbg Api(s) resides in example\kernel.. you know the project for sure

You need to use (import) the HyperLog.dll project in your project and then you can use the LogInfo functions and macros as well as exporting (giving) a pointer to the hyperhv.dll at the initialization time. (The way that hyperkd.sys passes the pointer to these functions to hyperhv.dll).

Or another option is importing HyperLog.dll to hyperhv.dll project and changing the code for initialization callbacks. This is also and option but in this option, you will not have access to LogInfo in your top-level project.

okay to understand better

/
* @file Loader.c
* @author Sina Karvandi (sina@hyperdbg.org)
* @brief The functions used in loading the VMM and RM
* @version 0.2
* @date 2023-01-29
*
* @copyright This project is released under the GNU Public License v3.
*
*/
#include "pch.h"

/
* @brief Initialize the VMM and Reversing Machine
*
* @return BOOLEAN
*/
BOOLEAN
LoaderInitVmmAndReversingMachine()
{
MESSAGE_TRACING_CALLBACKS MsgTracingCallbacks = {0};
VMM_CALLBACKS VmmCallbacks = {0};

//
// Allow to server IOCTL
//
g_AllowIOCTLFromUsermode = TRUE;

//
// Fill the callbacks for the message tracer
//
MsgTracingCallbacks.VmxOperationCheck = VmFuncVmxGetCurrentExecutionMode;
// MsgTracingCallbacks.CheckImmediateMessageSending = KdCheckImmediateMessagingMechanism;
// MsgTracingCallbacks.SendImmediateMessage = KdLoggingResponsePacketToDebugger;

//
// Fill the callbacks for using hyperlog in VMM
//
VmmCallbacks.LogCallbackPrepareAndSendMessageToQueueWrapper = LogCallbackPrepareAndSendMessageToQueueWrapper;
VmmCallbacks.LogCallbackSendMessageToQueue = LogCallbackSendMessageToQueue;
VmmCallbacks.LogCallbackSendBuffer = LogCallbackSendBuffer;
VmmCallbacks.LogCallbackCheckIfBufferIsFull = LogCallbackCheckIfBufferIsFull;

//
// Fill the VMM callbacks
//
// VmmCallbacks.VmmCallbackTriggerEvents = DebuggerTriggerEvents;

//
// Initialize message tracer
//
if (LogInitialize(&MsgTracingCallbacks))
{
//
// Initialize Vmx
//
if (VmFuncInitVmm(&VmmCallbacks))
{
LogDebugInfo("HyperDbg's hypervisor loaded successfully");

//
// Initialize the debugger
//
if (CoreInitReversingMachine())
{
LogDebugInfo("HyperDbg's reversing machine loaded successfully");

//
// Set the variable so no one else can get a handle anymore
//
g_HandleInUse = TRUE;

return TRUE;
}
else
{
LogError("Err, HyperDbg's debugger was not loaded");
}
}
else
{
LogError("Err, HyperDbg's hypervisor was not loaded");
}
}
else
{
LogError("Err, HyperDbg's message tracing module was not loaded");
}

//
// Not loaded
//
g_AllowIOCTLFromUsermode = FALSE;

return FALSE;
}

/**
* @brief Uninitialize the log tracer
*
* @return VOID
*/
VOID
LoaderUninitializeLogTracer()
{
LogDebugInfo("Unloading HyperDbg's debugger...\n");

#if !UseDbgPrintInsteadOfUsermodeMessageTracking

//
// Uinitialize log buffer
//
LogDebugInfo("Uninitializing logs\n");
LogUnInitialize();
#endif
}

the is the portion of code in the hyperdbg_driver project

i think it implement what you are telling me right now if im not wrong?

all i have to do is to enable the the WPP tracing in the project like in hypervisor from scrath tutuoriel right?

yes, it's an example implementation.

Nope, WPP tracing is not used in HyperDbg. Instead we wrote a custom project (which is called HyperLog.dll) for that.

You don't need to use WPP tracing.

Ok got it bro i will use this example as a start a
To read the output of the loginfo stuffs what i should do in this case ?and sorry for sure for flooding you with tons of questions its just the project is so big to read and I'm trying to focus currently on the memory introspection features not the logs stuffs ...

No worries, ask all of your questions.

Once you have hyperlog.dll project up and running, you need to have a user-mode thread (separate from original thread) that listens for messages and possible commands from VMX-root mode. It shows all of the messages you sent using LogInfo or other variants (LogWarning, LogError, etc.). I think the user-mode example of the examples implements such a behavior.

guys, anyone faced issues while using MemoryMapperReadMemorySafeByPhysicalAddress

it works well until i try to use it to read the content of a Page Directory Entry (PDE)

UINT64
GetPTAddress(UINT64 PDEAddress)
{
UINT64 PDE = 0;
if (MemoryMapperReadMemorySafeByPhysicalAddress(PDEAddress, &PDE, 8) == STATUS_SUCCESS)
{
if (PDE & 0x1)
{ // check if the PDE is valid
return (PDE & (0x00000000FFFFFF000));
}
else
{
LogInfo("PDE is invalid , the current content of PDE is 0x%llx ",PDE);
return 0;
}
}
LogInfo("MemoryMapperReadMemorySafeByPhysicalAddress returned 0 ...");
return 0;
}

MemoryMapperReadMemorySafeByPhysicalAddress in this case return FALSE

what is this mask here? 0x00000000FFFFFF000

pfn is larger than that

0x00000000FFFFFF000 this mask is used to extract the address of the page table from the PDE

Oh, you got 36 bit pfns, ok

Yes exactly bro

Are you sure the page-table is mapped in your target process? In VMX root-mode, we always stick to the system process (PID=4).

Oh, wait it's a physical address 🤔

Did you actually check that it is not a large (2mb) page?

Windows does use them a lot

Check the PS bit in the PDE

no its not a large page (2mb) i checked the translation using !pte in windbg after exiting hyperdbg

yes its a physical address , i tried to read the content of the memory pointed by that address to extract the address of the next stage in the address translation (aka page table address)

its weird MemoryMapperReadMemorySafeByPhysicalAddress works fine with PML4E, PDPE

but with PDE no

What's the output of !pte ?

wait

here is the output of my custom hyperdbg code

- The Physical Address of the Page Map Level entry (PML4E) associated to the Virtual address 0x2483cb5000 of the processs 0xffffe38500000284 is 0x26b61000
- The Physical Address of Page Directory Pointer entry (PDPE) associated to the Virtual address 0x2483cb5000 of the processs 0xffffe38500000284 is 0x1a475490
- The Physical Address of Page Directory entry (PDE) associated to the Virtual address 0x2483cb5000 of the processs 0xffffe38500000284 is 0x26c760f0
- (GetPTAddress:116) | MemoryMapperReadMemorySafeByPhysicalAddress returned 0

Why windbg? 🤦‍♂
HyperDbg itself has a nice !pte.

https://docs.hyperdbg.org/commands/extension-commands/pte

It does not show PTE contents

😅its just my code is a miss now ( a mixte between hyperdbg code and my shitty code) so i tried to make a clean exit to works without doubt , but i already know that hyperdbg !pte command works perfect

0x1a475490 - have you forgot to zero lowest 12 bits? How can va 0x2483cb5000 map to 0x1a475490 pa?

2: kd> !process 0 0 HelloWorld.exe
PROCESS ffffcf0aba747080
SessionId: 1 Cid: 0284 Peb: 2483cb5000 ParentCid: 10ac
DirBase: 26b61002 ObjectTable: ffff960306a14900 HandleCount: 40.
Image: HelloWorld.exe

2: kd> !vtop 26b61000 2483cb5000
Amd64VtoP: Virt 0000002483cb5000, pagedir 0000000026b61000
Amd64VtoP: PML4E 0000000026b61000
Amd64VtoP: PDPE 000000001a475490
Amd64VtoP: PDE 0000000026c760f0
Amd64VtoP: PTE 0000000026b775a8
Amd64VtoP: Mapped phys 0000000170078000
Virtual address 2483cb5000 translates to physical address 170078000.

I don't get this output tbh

and here is the output of windbg

Use !pte please

Oh, that means it's an entry address I guess

0x1a475490 is the physical address of PDPE, then i will use MemoryMapperReadMemorySafeByPhysicalAddress to read its content to extract the address of Page directory table

if you compare the results of my code and the output of !vtop i was doing fine 😂

Can you use !pte instead of !vtop ?

And show the output

okay

!pte of windbg or hyperdbg im confused xd

windbg

No offense to @HughEverett , it's just a habit :)

1: kd> !pte 2483cb5000
VA 0000002483cb5000
PXE at FFFF8E472391C000 PPE at FFFF8E4723800490 PDE at FFFF8E47000920F0 PTE at FFFF8E001241E5A8
contains 8A0000001A475867 contains 0A00000026C76867 contains 0A00000026B77867 contains 8100000170078847
pfn 1a475 ---DA--UW-V pfn 26c76 ---DA--UWEV pfn 26b77 ---DA--UWEV pfn 170078 ---D---UW-V

next time we should use !pte hyperdbg 😅or sina will be angry xd

Agreed

I feel threatened :)

Well, looks like a totally normal mapping, can you just compare the pte content output?

well i told based on the analysis of the output of !pte command and output of my "custom " address translation code they are the same

except that i cannot go further in the PDE stage when i need to read the content of PDE to get the physical address of next table

so why "MemoryMapperReadMemorySafeByPhysicalAddress" doesnt work with PDE as input is the big question now xd

I can't even see where it can return FALSE

So I blame you xD

Wait dude, BOOLEAN MemoryMapperReadMemorySafeByPhysicalAddress

It returns BOOLEAN

And you compare it with status_success?

yes

exactly bro it returns BOOLEAN

UINT64
GetPTAddress(UINT64 PDEAddress)
{
UINT64 PDE = 0;
if (MemoryMapperReadMemorySafeByPhysicalAddress(PDEAddress, &PDE, 8) == STATUS_SUCCESS)
{
if (PDE & 0x1)
{ // check if the PDE is valid
return (PDE & (0x00000000FFFFFF000));
}
else
{
LogInfo("PDE is invalid , the current content of PDE is 0x%llx ",PDE);
return 0;
}
}
LogInfo("MemoryMapperReadMemorySafeByPhysicalAddress returned 0 ...");
return 0;
}

So why do you compare it with ntstatus?

🥲

ouuuuuuuuuuuuuuuuups

nice remark bro

Well I can only recommend to have more sleep in these cases

😮‍💨 🤣

you got my point xd

hypervisor stuffs need more sleep hour

hours*

🙃😂

sorry guys for this bug 🤣

No problem

guys im following this strategy to test my code and correct me give me advices if im wrong ... so since i dont want to make things complicated from the start im trying to use one single core for testing to avoid any synchro issues in my code .. however when i do that the machine become "halt" not responding (with only 4 or 5 EPT_HOOK_MONITOR) should i assume that something is wrong in my code (i modified slightly mft handler and ept violation handler) or its just 1 core is not enough to handle hyperdbg code

windows 10

Well, monitor works on both single core and multiple core. The problem is most likely on your MTF modifications. 🤔

Yes bro that's what i found xd it , i added a loginfo("hello word1") at the start of mft handler and loginfo("hello world2") at the start of ept violation handler then I start seeing an infinite loop lf "hello world1" in the output which is weird since in theory the number of "hello world1" should be approximately the same of "hello world 2"

Xd

Well I will make some code review and verify

@HughEverett I fixed the bug😂😐 finally, let's say I confused the pid of the parent process with the child a very long story

Btw guys , any idea /scientific metrics about the minimum number of vm exits per second that we can consider an indicator of a slow VM?

It's all empirical, since different CPU gens and uarchs have different vmexit latencies.

I know but we cannot define let's say sole standard number for "average cpu " that widely used

I don't think so

i discovered that the system freeze if we put rw monitor hooks on physical addresses of PML4E, PDPE, PDE, PTE of the peb address of some process

can someone replicate the same behavior ?

to be more specific the system freez "sometimes" and sometimes it continue working

hyperdbg 0.13

I encountered this problem a few years ago. The problem is not because of the HyperDbg's implementation. My conclusion was that there are some bits (like dirty bit in the page table) that cause a VM-exit once they're applied by CPU. But, we couldn't re-apply them to the memory as they don't have an instruction for applying the dirty bit and it's the processor that applies it. I think the reason why HyperDbg hangs is because of this (but I'm not sure and I couldn't verify the assumption).

The 'dirty' bit is an example, there might be other bits as well.

Can you elaborate more on this dirty bit problem? I didn't catch the exact problem

Dirty bit don't cause VM-exits

mmmmmmm my assumption was that probably the PML4 table is heavily accessed since thats the start point of translating addresses and when you try to put any monitor point on some specific PML4E it will end up triggering any read/write in the table since its seize is exactly 0x1000 bytes

Got it, EPT violation due to dirty bits update?

Do you actually get VM-exits logged on PML4 lookups? Because those are supposed to be cached in TLB all the time

But that's interesting, never thought of this use case

well each time we discover some strange behavior in virtualization xd

thats a strange world

Well, too bad :)

i have modified the code of the MFT handler by adding a debug message : LogInfo("Inside the MonitorTrapFlag handler again ");

at the start of EptHookHandleMonitorTrapFlag

and when i applied the monitor rw hook on the PML4E it generate an infinite loop of MFT exit events

Oh, I got it, tlb cache entries actually hold ept rwxu bits, so even if they are cached in tlb, they are still consulted for a possible ept violation exit

in other words the ept violation will happen for sure right

yeah, no matter if it's cached or fetched from the memory

in fact if it's not in the cache, it is fetched into the cache first

so consulted with the cache entry anyway

got your point bro it seems logic

but what i dont understand now

is that i modified the EPT hook handler function

EptHookHandleHookedPage

in such way that it will display the details of the access

like : LogInfo("Guest RIP : 0x%llx tries to read the page at : 0x%llx", HvGetRip(), PhysicalAddress);

however i dont see this output in the current logs displayed

what's in the logs?

wait i will make some screen and show you

Obviously instruction fetch also causes ept violation btw, so every instruction would cause a EPT violation

Also, if you need to monitor page table changes, why don't you trap just writes?

Monitoring all page table accesses doesn't make sense

in order to minitor the write only what config i need to set up

because if im not wrong you cannot set up write alone?

EPT protection definitely allows write-only, as well as execute-only on later (IVB+) cpus

really?

because if im not wrong the current hyperdbg implementation

add always the read

even if the user specified the write only

Oh, that's hyperdbg specifics then )

I'm just an Intel guy

im aware of the execute only

but for the write only

thats the first time i hear about

@HughEverett i think sina have more details about the implementation

Applicable for every pte type

well if that was the case

i think we should have a fork of hyperdbg that ignore read access when the user specify write access only

this will help really improve the performance since usually we are intersted in write access events from a malware detection perspective

yes

what is the exit reason? can you print that?

Do you mean that each translation of a page (VA to PA), will create EPT violation?

That's what I deduce from the the discussion, it makes sense. Though I haven't tried myself

many of the addresses that caused the EPT violation are related to the memory manager code

Ohh, but that's a manual page walk with an actual pte access

HyperDbg allows all of the possible combination. The only thing that is not allowed is a page that you want to intercept its reads but not writes. (It automatically sets the write to FALSE) because it's not supported in Intel processors (you'll get a VM error if you configure it like this).

You can't do it, Intel won't allow it. That's why HyperDbg automatically configures the (read to FALSE) bit.

Got it. That's reasonable.

so

in order to monitor write access

only

we should

MonitorDetails.SetHookForRead = FALSE;
MonitorDetails.SetHookForWrite = TRUE;
MonitorDetails.SetHookForExec = FALSE;

Not possible, using this combination is not allowed in Intel.

so something like

MonitorDetails.SetHookForRead = FALSE;
MonitorDetails.SetHookForWrite = TRUE;
MonitorDetails.SetHookForExec = TRUE;

again, not possible.

He needs a EPT violation only on writes

That means R is 1, but W is 0

In EPT

You can't set hook for write without having it for reads.

You can

R = 0, W = 1 is invalid

But it's the other way round

R = 1, W = 0

well, let me double check the code. I think we're mixing set write to FALSE with set hook to FALSE.

Oh, yes

I mean this one, but in terms of setting hook, it's NOT(Page bits).

So, what's the problem Poikourditchy78 ? HyperDbg certainly support this one on its current design.

im just confused about the

defintion of access from code perspective

so im not wrong

MonitorDetails.SetHookForRead = TRUE;
MonitorDetails.SetHookForWrite = FALSE;
MonitorDetails.SetHookForExec = TRUE;

will trigger write access only

Setting hook = Not (Setting RWX bits)

okaaay

No, you need to configure it like:

MonitorDetails.SetHookForRead = FALSE;
MonitorDetails.SetHookForWrite = TRUE;
MonitorDetails.SetHookForExec = FALSE;

so : not setting hook = Setting RWX bits)

got it got it got it

yes

mmmmm however we should solve the puzzle of this PML4E xd

i found this article

https://hypervsir.blogspot.com/2014/11/monitor-trap-flag-mtf-usage-in-ept.html

it somehow implements the same idea to monitor page tables (he will set page table entries to "read only" according to him"