Purpose

CPU isolation (or CPU affinity) is a way to force the system scheduler to only use some of the logical CPU cores for a particular process (and all sub-processes it creates). If isolation is not utilized the system will try to balance the work and use all cores for all the processes.
The idea here is that in audio playback we do not want to balance the entire workload equally - some processes are much more important to us than others. We want these processes to have enough CPU cycles and we also do not want to switch processor context and lose L1 and L2 cache advantage.
Having enough cores dedicated to the most important processes ensures low latency and smooth flow of playback data which can only be beneficial to SQ.
Again, how much improvement you can gain depends on many factors:

• playback system in use (Stylus, Roon, HQPlayer...) 
• number of cores you have available 
• do you have hyperthreading enabled or not 
• the maximum CPU frequency you are using
• song samplerate/format you usually play

The main thing is that this is very easy to try (even mid-playback) so you have little to lose (risk level is relatively low). 

Usage

Core isolation is specified by listing core numbers (starting from zero) separated by comma or by specifying range using hyphen: "0,1,2,3" is therefore equivalent to "0-3".

The format of the CPU isolation line in Euphony is to first specify isolation for all processes and then you specify isolation for specific processes. The format is shown in placeholder text:

<default-iso> <program-name1> <iso1> <program-name2> <iso2> ....

Here is an actual example for a CPU with 8 cores:

0-1 stylus 2-3 gstp 4-7

All processes are assigned to cores 0,1
stylus process is assigned to cores 2,3
gstp process is assigned to cores 4,5,6,7

For a user to be able to play with this, it is important to know which processes are important:

• stylus - Euphony web gui application
• gstp - Euphony player process
• RoonServer - all Roon processes can be targeted with this string 
• RoonBridge - Roon bridge endpoint
• hqplayerd - HQPlayer embedded
• networkaudiod - HQplayer NAA process

The processes listed above are relevant for music playback - there are other system processes listed when you get isolation info but those use minimal CPU resources and are not usually worth bothering with. 

Only stylus is always active, other processes appear depending on the audio system selected.

To be able to select a process, full process name is not needed - if you use "Roon" you can target all Roon core processes and Roon bridge processes - your entered string just needs to be a part of the full process name.

When you click Apply the isolation will be instantly applied (and the list of processes and their set isolation will be shown).
If you erase the field content and click Apply then the current isolation will be shown again but nothing will change. If you want to reset to system default you have to enter the default isolation and cover all cores - in our example for CPU with 8 cores you enter just:

0-7

and click Apply.

Case is important: "Roon" will select every Roon process - "roon" will select nothing.

HQPlayer does its own isolation: HQPlayer's owner Jussi strongly advises against using Euphony isolation while HQPlayer is in use. Even if you don't mention the program explicitly in CPU isolation string it will still be covered by default isolation.
The only way to do this with the least amount of interference with HQPlayer's own isolation is to explicitly assign hqplayerd process to all cores by adding: "hqplayerd 0-7" (if you have 8 cores) to whatever isolation for other processes you came up with. This should ideally be done even before HQPe playback starts. It is also important to note which cores HQPe prefers using - open CPU/Temp dialog while HQPe playback is active and note which cores show most CPU usage - these are most probably the cores that HQPe uses for upsampling algorithms. Build the rest of the CPU isolation around it - don't use these cores for other processes.

For audio playback, the most important processes are those that handle music data - for Stylus playback this is gstp process - responsible for decoding audio streams and sending music bytes to the audio card.

How many cores to add to each process? Which processes can be grouped on the same cores and which need to be alone? This must just be experimented with. In general, for Stylus playback, gstp should be given more cores than any other process. 

Risk level

In principle, you cannot do much damage with CPU isolation. In an extreme case you can isolate everything to 1 core and if you also have very low max CPU frequency you can make your control of everything very slow - maybe to the point of unusability but in most cases you are pretty safe.

Advanced CPU isolation: IRQ handling

In recent version (20210217) a hidden feature was added to CPU isolation possibilities - isolation of IRQs.

To take advantage of this feature some background knowledge is necessary.

IRQ stands for Interrupt request. Interrupt request will momentarily stop the CPU from doing whatever it is doing at the moment and demand execution of the interrupt handler code.
There are hardware interrupts (user pressed a key, network card received data...) and software interrupts (fired by some precise timer to synchronize things - often video or music playback).  

Interrupt handlers must be coded to be extremely fast and to return control as soon as possible. To be able to do that, if there is any real work to be done, this work is not done in interrupt handler code itself. Interrupt handler will only move the relevant interrupt data into a temporary buffer and return control immediately. The actual work on this data is done in the corresponding interrupt handler kernel thread. Interrupt handler kernel threads are kernel background processes that are always running and process the data as it becomes available after interrupt handler serves an interrupt request. 

Which CPU will handle which interrupt is determined initially by the system itself - IRQ with a certain number will be always served by a particular CPU - so IRQs have inherent CPU affinity. But, like process CPU affinity, there is a way to change IRQ CPU affinity. 

It is crucial here to note that interrupt is handled in 2 steps: first step is interrupt handler handling the data by only copying it to some temporary buffer and the second step is the processing of that data by the corresponding IRQ handler kernel thread.
On which CPU will these two tasks be executed?
IRQ handler will execute on the CPU it has affinity set for.
IRQ handler kernel thread will be executed according to its process affinity.

Now, so far with CPU isolation we have only addressed affinity of processes and by doing so, with the default isolation for all processes we have also changed isolation of all IRQ handler kernel threads!
While initially, before applying any CPU isolation, IRQ handlers and IRQ handler kernel thread were probably executing on the same CPU, by applying default isolation we changed affinity of those kernel threads and possibly inadvertently separated them from their IRQ handlers! This certainly cannot help but how much effect it has is hard to tell without some experimentation.

That is why we provided a way to change IRQ affinity and their kernel threads affinity together.

Again, like with processes - not all IRQs are equally important - we are mostly interested in those that handle audio data, whether it comes via network or is sent to the audio card (with emphasis on the latter). 

To be able to do anything meaningful one must identify IRQ numbers.

To help with that you can look at the output of "cat /proc/interrupts" command by typing the period sign while in CPU isolation field.
Something like this will show in a dialog:

             CPU0       CPU1       CPU2       CPU3

   0:         15          0          0          0  IR-IO-APIC    2-edge      timer

   8:          0          0          0          1  IR-IO-APIC    8-edge      rtc0

   9:          0          4          0          0  IR-IO-APIC    9-fasteoi   acpi

  10:          0      33922          0          0  IR-IO-APIC   10-edge      ite-cir

  14:          0          0          0          0  IR-IO-APIC   14-fasteoi   INT344B:00

  16:          0          0          0          0  IR-IO-APIC   16-fasteoi   i801_smbus

  20:          0          0          0          0  IR-IO-APIC   20-fasteoi   idma64.0

  23:          0          0          0          0  IR-IO-APIC   23-fasteoi   mmc0

 120:          0          0          0          0  DMAR-MSI    0-edge      dmar0

 121:          0          0          0          0  DMAR-MSI    1-edge      dmar1

 122:          0          0     342845          0  IR-PCI-MSI 376832-edge   ahci[0000:00:17.0]

 123:          0          0 2308292858       1325  IR-PCI-MSI 327680-edge      xhci_hcd

 124:          0          0          0         37  IR-PCI-MSI 360448-edge      mei_me

 125:          0   35819585          0          0  IR-PCI-MSI 520192-edge      eth0 

 127:          0          0          0        200  IR-PCI-MSI 32768-edge      i915

In the first column are the IRQ numbers - we need to remember numbers that are of interest to us.
In the following several columns are counts of interrupts served by the corresponding CPU since startup (each CPU has its own column).
Although some interrupts are easy to identify - IRQ 125 is a network card (interface eth0 - last column contains the name), others may be more cryptic.
Since most people connect their DAC through USB it is worth knowing that the name associated with USB is usually xhci_hcd. But even if you didn't know that - just by the very large count of interrupts on IRQ 123 on CPU2 and knowing that this machine did a lot of music playback since startup we can be sure that IRQ 123 is handling our audio data passing through USB.

For example, to set isolation to core 3 for our audio IRQ 123 you have to add to CPU isolation line the following string: irq/123 3

So the syntax for targeting specific IRQ handler and corresponding IRQ handler kernel thread is simple: irq/<IRQ_NUM> (irq, slash, IRQ number). This way both the handler and its kernel thread will be on the same CPU(s).

If in isolation string field there is even one IRQ specified, then when you click Apply in the resulting list of processes all IRQ kernel thread processes will be listed too. Otherwise, these processes and their isolation will not be visible on the list. 

One thing you have to keep in mind: IRQ numbers are not always constant! That means the same IRQ numbers are not always paired with the same device across boots! They do not shuffle randomly - most of the time they will be the same but occasionally two devices may swap places. If one of the devices is not important you can group it with the important one so the occasional swap will have no effect or if both are important then you probably have already assigned them each to the sole individual core so the swap again will not change anything. In any case, you need to occasionally check if your isolation is as you expected it to be. 

Do I even want to change the default IRQ affinity?
Is it that important for IRQ handler and its kernel thread to execute on the same CPU?
Should processes that handle audio data (like gstp) be on the same CPU as audio IRQ?
How much gain in SQ (if any) is to be expected from optimizing this?

We do not have any answers for you ready. The development team cooked this up but the listening team has had no time yet to experiment extensively with it.

However, initial reports from several users seem promising. It is obviously system dependent but there appears to be a clear improvement from adding IRQ isolation to already established process isolation. It should also be noted that aside from deciding where to put USB IRQ you should also consider managing all other IRQs that show any significant traffic (count) and probably spread them away from audio path related IRQs.