moonpower/P3DS-test
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Memory rework pt 2 (#801)

Browse Source 
* Memory: Rework FCRAM management entirely

Disables a lot of functionality... but I didn't want to commit too much to this commit
Also reworks virtual memory management somewhat (but needs more work)

* Accurately handle MemoryState for virtual memory

Previously all non-free blocks were marked as Reserved

* Memory: Consolidate state and permission changes

Can now use a single function to change either state, permissions, or both
Also merge vmem blocks that have the same state and permissions

* Memory: Fix double reset for FCRAM manager

Fix minor bug with permission tracking

* Memory: Implement Protect operation in ControlMemory

* Memory: Implement Unmap in ControlMemory

Also do a sanity check to make sure the memory region is free for linear allocations

* Memory: Make TLS only 0x200 bytes for each thread

Also move TLS to Base region

* RO: Unmap CROs when unloaded

Thanks @noumidev

* Kernel: Return used app memory for Commit ResourceLimit

Not quite correct, but nothing to be done until process management is improved
Also remove the stack limit for CXIs (thanks amogus)

* Kernel: Report used app memory for GetProcessInfo 2

Not really correct, but it should be accurate for applications at least

* Formatting changes

* Initial fastmem support

* PCSX2 fastmem depression

* Move away from PCSX2 fastmem

* Add enum_flag_ops.hpp

* Finally building on Windows

* Almost got a PoC

* Fix arm64 builds

* This somehow works

* This also works...

* Properly fix fastmem

* Add free region manager

* Update boost

* Add ScopeExit

* Comment out asserts on Linux/Mac/Android

* Comment out ASSERT_MSG asserts too

* Fix derp

* Attempt to fix Android

* Disable fastmem on Android

* Fix Android again maybe pt 2

* android pls

* AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

* AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

* Update host_memory.cpp

* Properly reset memory arena on reset

* Proper ashmem code for Android

* more

* Add temporary Android buildjet script for faster prototype builds

* Fix fastmem (again)

* Clean up shared memory

* Remove Android BuildJet runner

* a

* Revert "a"

This reverts commit 5443ad6f2a794c19c9b1a1567ca1c7f58eed78cd.

* Re-add ELF support

* Re-add 3DSX support

* GetSystemInfo, GetProcessInfo: Memory sizes should be in bytes

* Update Boost

* Update metal-cpp

* Fix metal renderer compilation

* Fix fastmem mapping

* Clean up fastmem code

* Fix oopsie again

* Emulator: Reorder struct

* Kernel types: Cleanup

* Cleanup

* More cleanup

* Make invalid mprotects warn instead of panicking

* Add setting for toggling fastmem

* More cleanup

* Properly initialize BSS to zeroes

* Remove unused code

* Formatting

* Cleanup

* Memory/CRO: Workaround for Pokemon XY

* NCSD loader: Fix BSS (again)

* NCSD loader: Fix BSS (again) (again)

* More memory fixes

* Memory: Remove unused code

* FS: Warn on unimplemented functions instead of panic

* Update software_keyboard.cpp

* Libretro: Add fastmem option

* FRD: Stub SaveLocalAccountData

---------

Co-authored-by: PSI-Rockin <PSI-Rockin@users.noreply.github.com>
This commit is contained in:
wheremyfoodat
2025-08-07 20:18:09 +03:00
committed by GitHub
parent 5ebee8ea72
commit 6d1ef7cb4f
40 changed files with 2901 additions and 526 deletions
Download Patch File Download Diff File Expand all files Collapse all files

63
include/kernel/fcram.hpp Normal file
View File

@@ -0,0 +1,63 @@
#pragma once
#include <list>
#include <memory>
#include "helpers.hpp"
class Memory;
enum class FcramRegion {
App = 0x100,
Sys = 0x200,
Base = 0x300,
};
struct FcramBlock {
u32 paddr;
s32 pages;
FcramBlock(u32 paddr, s32 pages) : paddr(paddr), pages(pages) {}
};
using FcramBlockList = std::list<FcramBlock>;
class KFcram {
struct Region {
struct Block {
s32 pages;
s32 pageOffset;
bool used;
Block(s32 pages, u32 pageOffset) : pages(pages), pageOffset(pageOffset), used(false) {}
};
std::list<Block> blocks;
u32 start;
s32 pages;
s32 freePages;
public:
Region() : start(0), pages(0) {}
void reset(u32 start, size_t size);
void alloc(std::list<FcramBlock>& out, s32 pages, bool linear);
u32 getUsedCount();
u32 getFreeCount();
};
Memory& mem;
Region appRegion, sysRegion, baseRegion;
uint8_t* fcram;
std::unique_ptr<u32> refs;
public:
KFcram(Memory& memory);
void reset(size_t ramSize, size_t appSize, size_t sysSize, size_t baseSize);
void alloc(FcramBlockList& out, s32 pages, FcramRegion region, bool linear);
void incRef(FcramBlockList& list);
void decRef(FcramBlockList& list);
u32 getUsedCount(FcramRegion region);
};

4
include/kernel/kernel.hpp
View File

@@ -7,6 +7,7 @@
#include <vector>
#include "config.hpp"
#include "fcram.hpp"
#include "helpers.hpp"
#include "kernel_types.hpp"
#include "logger.hpp"
@@ -25,6 +26,8 @@ class Kernel {
CPU& cpu;
Memory& mem;
KFcram fcramManager;
// The handle number for the next kernel object to be created
u32 handleCounter;
// A list of our OS threads, the max number of which depends on the resource limit (hardcoded 32 per process on retail it seems).
@@ -247,6 +250,7 @@ class Kernel {
}
ServiceManager& getServiceManager() { return serviceManager; }
KFcram& getFcramManager() { return fcramManager; }
Scheduler& getScheduler();
void sendGPUInterrupt(GPUInterrupt type) { serviceManager.sendGPUInterrupt(type); }

222
include/kernel/kernel_types.hpp
View File

@@ -1,93 +1,108 @@
#pragma once
#include <array>
#include <cstring>
#include "fs/archive_base.hpp"
#include "handles.hpp"
#include "helpers.hpp"
#include "result/result.hpp"
enum class KernelObjectType : u8 {
AddressArbiter, Archive, Directory, File, MemoryBlock, Process, ResourceLimit, Session, Dummy,
// Bundle waitable objects together in the enum to let the compiler optimize certain checks better
Event, Mutex, Port, Semaphore, Timer, Thread
AddressArbiter,
Archive,
Directory,
File,
MemoryBlock,
Process,
ResourceLimit,
Session,
Dummy,
// Bundle waitable objects together in the enum to let the compiler optimize certain checks better
Event,
Mutex,
Port,
Semaphore,
Timer,
Thread
};
enum class ResourceLimitCategory : int {
Application = 0,
SystemApplet = 1,
LibraryApplet = 2,
Misc = 3
Application = 0,
SystemApplet = 1,
LibraryApplet = 2,
Misc = 3,
};
// Reset types (for use with events and timers)
enum class ResetType {
OneShot = 0, // When the primitive is signaled, it will wake up exactly one thread and will clear itself automatically.
Sticky = 1, // When the primitive is signaled, it will wake up all threads and it won't clear itself automatically.
Pulse = 2, // Only meaningful for timers: same as ONESHOT but it will periodically signal the timer instead of just once.
OneShot = 0, // When the primitive is signaled, it will wake up exactly one thread and will clear itself automatically.
Sticky = 1, // When the primitive is signaled, it will wake up all threads and it won't clear itself automatically.
Pulse = 2, // Only meaningful for timers: same as ONESHOT but it will periodically signal the timer instead of just once.
};
enum class ArbitrationType {
Signal = 0,
WaitIfLess = 1,
DecrementAndWaitIfLess = 2,
WaitIfLessTimeout = 3,
DecrementAndWaitIfLessTimeout = 4
Signal = 0,
WaitIfLess = 1,
DecrementAndWaitIfLess = 2,
WaitIfLessTimeout = 3,
DecrementAndWaitIfLessTimeout = 4,
};
enum class ProcessorID : s32 {
AllCPUs = -1,
Default = -2,
AppCore = 0,
Syscore = 1,
New3DSExtra1 = 2,
New3DSExtra2 = 3
AllCPUs = -1,
Default = -2,
AppCore = 0,
Syscore = 1,
New3DSExtra1 = 2,
New3DSExtra2 = 3
};
struct AddressArbiter {};
struct ResourceLimits {
HorizonHandle handle;
HorizonHandle handle;
s32 currentCommit = 0;
s32 currentCommit = 0;
};
struct Process {
// Resource limits for this process
ResourceLimits limits;
// Process ID
u32 id;
// Resource limits for this process
ResourceLimits limits;
// Process ID
u32 id;
Process(u32 id) : id(id) {}
Process(u32 id) : id(id) {}
};
struct Event {
// Some events (for now, only the DSP semaphore events) need to execute a callback when signalled
// This enum stores what kind of callback they should execute
enum class CallbackType : u32 {
None, DSPSemaphore,
};
// Some events (for now, only the DSP semaphore events) need to execute a callback when signalled
// This enum stores what kind of callback they should execute
enum class CallbackType : u32 {
None,
DSPSemaphore,
};
u64 waitlist; // A bitfield where each bit symbolizes if the thread with thread with the corresponding index is waiting on the event
ResetType resetType = ResetType::OneShot;
CallbackType callback = CallbackType::None;
bool fired = false;
u64 waitlist; // A bitfield where each bit symbolizes if the thread with thread with the corresponding index is waiting on the event
ResetType resetType = ResetType::OneShot;
CallbackType callback = CallbackType::None;
bool fired = false;
Event(ResetType resetType) : resetType(resetType), waitlist(0) {}
Event(ResetType resetType, CallbackType cb) : resetType(resetType), waitlist(0), callback(cb) {}
Event(ResetType resetType) : resetType(resetType), waitlist(0) {}
Event(ResetType resetType, CallbackType cb) : resetType(resetType), waitlist(0), callback(cb) {}
};
struct Port {
static constexpr u32 maxNameLen = 11;
static constexpr u32 maxNameLen = 11;
char name[maxNameLen + 1] = {};
bool isPublic = false; // Setting name=NULL creates a private port not accessible from svcConnectToPort.
char name[maxNameLen + 1] = {};
bool isPublic = false; // Setting name=NULL creates a private port not accessible from svcConnectToPort.
Port(const char* name) {
// If the name is empty (ie the first char is the null terminator) then the port is private
isPublic = name[0] != '\0';
std::strncpy(this->name, name, maxNameLen);
}
Port(const char* name) {
// If the name is empty (ie the first char is the null terminator) then the port is private
isPublic = name[0] != '\0';
std::strncpy(this->name, name, maxNameLen);
}
};
struct Session {
@@ -145,92 +160,90 @@ struct Thread {
};
static const char* kernelObjectTypeToString(KernelObjectType t) {
switch (t) {
case KernelObjectType::AddressArbiter: return "address arbiter";
case KernelObjectType::Archive: return "archive";
case KernelObjectType::Directory: return "directory";
case KernelObjectType::Event: return "event";
case KernelObjectType::File: return "file";
case KernelObjectType::MemoryBlock: return "memory block";
case KernelObjectType::Port: return "port";
case KernelObjectType::Process: return "process";
case KernelObjectType::ResourceLimit: return "resource limit";
case KernelObjectType::Session: return "session";
case KernelObjectType::Mutex: return "mutex";
case KernelObjectType::Semaphore: return "semaphore";
case KernelObjectType::Thread: return "thread";
case KernelObjectType::Dummy: return "dummy";
default: return "unknown";
}
switch (t) {
case KernelObjectType::AddressArbiter: return "address arbiter";
case KernelObjectType::Archive: return "archive";
case KernelObjectType::Directory: return "directory";
case KernelObjectType::Event: return "event";
case KernelObjectType::File: return "file";
case KernelObjectType::MemoryBlock: return "memory block";
case KernelObjectType::Port: return "port";
case KernelObjectType::Process: return "process";
case KernelObjectType::ResourceLimit: return "resource limit";
case KernelObjectType::Session: return "session";
case KernelObjectType::Mutex: return "mutex";
case KernelObjectType::Semaphore: return "semaphore";
case KernelObjectType::Thread: return "thread";
case KernelObjectType::Dummy: return "dummy";
default: return "unknown";
}
}
struct Mutex {
using Handle = HorizonHandle;
using Handle = HorizonHandle;
u64 waitlist; // Refer to the getWaitlist function below for documentation
Handle ownerThread = 0; // Index of the thread that holds the mutex if it's locked
Handle handle; // Handle of the mutex itself
u32 lockCount; // Number of times this mutex has been locked by its daddy. 0 = not locked
bool locked;
u64 waitlist; // Refer to the getWaitlist function below for documentation
Handle ownerThread = 0; // Index of the thread that holds the mutex if it's locked
Handle handle; // Handle of the mutex itself
u32 lockCount; // Number of times this mutex has been locked by its daddy. 0 = not locked
bool locked;
Mutex(bool lock, Handle handle) : locked(lock), waitlist(0), lockCount(lock ? 1 : 0), handle(handle) {}
Mutex(bool lock, Handle handle) : locked(lock), waitlist(0), lockCount(lock ? 1 : 0), handle(handle) {}
};
struct Semaphore {
u64 waitlist; // Refer to the getWaitlist function below for documentation
s32 availableCount;
s32 maximumCount;
u64 waitlist; // Refer to the getWaitlist function below for documentation
s32 availableCount;
s32 maximumCount;
Semaphore(s32 initialCount, s32 maximumCount) : availableCount(initialCount), maximumCount(maximumCount), waitlist(0) {}
Semaphore(s32 initialCount, s32 maximumCount) : availableCount(initialCount), maximumCount(maximumCount), waitlist(0) {}
};
struct Timer {
u64 waitlist; // Refer to the getWaitlist function below for documentation
ResetType resetType = ResetType::OneShot;
u64 fireTick; // CPU tick the timer will be fired
u64 interval; // Number of ns until the timer fires for the second and future times
bool fired; // Has this timer been signalled?
bool running; // Is this timer running or stopped?
u64 fireTick; // CPU tick the timer will be fired
u64 interval; // Number of ns until the timer fires for the second and future times
bool fired; // Has this timer been signalled?
bool running; // Is this timer running or stopped?
Timer(ResetType type) : resetType(type), fireTick(0), interval(0), waitlist(0), fired(false), running(false) {}
};
struct MemoryBlock {
u32 addr = 0;
u32 size = 0;
u32 myPermission = 0;
u32 otherPermission = 0;
bool mapped = false;
u32 addr = 0;
u32 size = 0;
u32 myPermission = 0;
u32 otherPermission = 0;
bool mapped = false;
MemoryBlock(u32 addr, u32 size, u32 myPerm, u32 otherPerm) : addr(addr), size(size), myPermission(myPerm), otherPermission(otherPerm),
mapped(false) {}
MemoryBlock(u32 addr, u32 size, u32 myPerm, u32 otherPerm)
: addr(addr), size(size), myPermission(myPerm), otherPermission(otherPerm), mapped(false) {}
};
// Generic kernel object class
struct KernelObject {
using Handle = HorizonHandle;
Handle handle = 0; // A u32 the OS will use to identify objects
void* data = nullptr;
KernelObjectType type;
Handle handle = 0; // A u32 the OS will use to identify objects
void* data = nullptr;
KernelObjectType type;
KernelObject(Handle handle, KernelObjectType type) : handle(handle), type(type) {}
KernelObject(Handle handle, KernelObjectType type) : handle(handle), type(type) {}
// Our destructor does not free the data in order to avoid it being freed when our std::vector is expanded
// Thus, the kernel needs to delete it when appropriate
~KernelObject() {}
// Our destructor does not free the data in order to avoid it being freed when our std::vector is expanded
// Thus, the kernel needs to delete it when appropriate
~KernelObject() {}
template <typename T>
T* getData() {
return static_cast<T*>(data);
}
template <typename T>
T* getData() {
return static_cast<T*>(data);
}
const char* getTypeName() const {
return kernelObjectTypeToString(type);
}
const char* getTypeName() const { return kernelObjectTypeToString(type); }
// Retrieves a reference to the waitlist for a specified object
// Retrieves a reference to the waitlist for a specified object
// We return a reference because this function is only called in the kernel threading internals
// We want the kernel to be able to easily manage waitlists, by reading/parsing them or setting/clearing bits.
// As we mention in the definition of the "Event" struct, the format for wailists is very simple and made to be efficient.
@@ -246,8 +259,7 @@ struct KernelObject {
case KernelObjectType::Timer: return getData<Timer>()->waitlist;
// This should be unreachable once we fully implement sync objects
default: [[unlikely]]
Helpers::panic("Called GetWaitList on kernel object without a waitlist (Type: %s)", getTypeName());
default: [[unlikely]] Helpers::panic("Called GetWaitList on kernel object without a waitlist (Type: %s)", getTypeName());
}
}
};
};