Metasploit x86/unicode_upper failed with "An encoding exception occurred"
I'm playing a bit with assembler and I decided to test out how metasploit encodes a payload into Unicode uppercase characters as assembly instructions which are meant to be decoded by itself. So I have written simple shell code consisting of 10 nops and a ret. I placed it into a file 1.payload.
I'm using metasploit:
Framework: 4.17.4-dev-4744fb01d9c4234a60c35dc78b9a8710fc19fa01
Console : 4.17.4-dev-4744fb01d9c4234a60c35dc78b9a8710fc19fa01
on Windows 10.
I tried to encode my payload like this (from cmd.exe):
msfvenom -a x86 --platform windows -p generic/custom
PAYLOADFILE=c:\test\1.payload -e x86/unicode_upper -o c:/out/encoded.bin
And I get this error:
Found 1 compatible encoders
Attempting to encode payload with 1 iterations of x86/unicode_upper x86/unicode_upper failed with An encoding exception occurred.
Error: An encoding exception occurred.
I do not have any zero bytes in my "payload" it's just 10 nop instructions, and a ret instruction. The encoder for alphs works fine but for some reason it generates weird non-Latin characters at the beginning of the output. Can someone help, do I use it right? How can I encode my x86 assembler payload in Unicode uppercase instructions?
