Escaping special characters inside bat file won't act the same

I spend the entire day looking for a solution but without any success. I have a weird bug with a bat file created to unzip 7zip files with. Here is the .bat file:

@echo off
setlocal enabledelayedexpansion

set "filename=%~2"
set "password=%~1"
echo Password after removing quotes is: !password!
echo Path: !filename!
"C:\Program Files\7-zip\7z.exe" x -y -o"C:\Users\XXXXXX\Downloads" !filename! -p!password!

From what I know you need to escape speciale characters inside .bat files with a "^". I have another C# script that do this before passing the string password to the .bat file, it looks like this:

static string EscapeSpecialCharacters(string input)
{
string pattern = @"[!&^]";
string escapeString = System.Text.RegulareExpressions.Regex.Replace(input, pattern, m=>m.Value == "&"?"^"+m.Value:"^^^"+m.Value);
return escapeString;
}

string escapedString = EscapeSpecialCharacters(originalString);
out_originalString = escapedString;

My problem happen when the password contain the character "^". For example, for password ABCDoE4vySI^K?Mwt!w^ will result into ABCDoE4vySI^^^^K?Mwt^^^!w^^^^ and the password will have this form inside the bat file ABCDoE4vySI^K?Mwt!w^ all good. For another string looking like this szEcal^q#Lc@JkYjAWda the string will get this form szEcal^^^^q#Lc@JkYjAWda but when it will get into the .bat file will have this form szEcal^^q#Lc@JkYjAWda, which will result in an error cand I can't understand why, it should be like this szEcal^q#Lc@JkYjAWda. Why the same case act two ways instead of respecting the same condition and act the same.

There is explained by the usage help of the Windows Command Processor cmd.exe output on running cmd /? in a command prompt window that a file name or any other argument string like a password must be enclosed in " on containing a space or one of these characters &()[]{}^=;!'+,`~ or a literally to interpret <>| which a password could contain too.

The second fact to consider is that with enabled delayed variable expansion a command line is parsed more than once. There should be read first: How does the Windows Command Interpreter (CMD.EXE) parse scripts? It explains in full details how cmd.exe parses a command line the first time.

The command line set "password=%~1" is parsed a second time after replacing %~1 by the first argument string passed to the batch file with surrounding " removed for delayed expanded variable references which means for one or more exclamation marks. A single exclamation mark is removed from the password string. If there are two exclamation marks the string between them is interpreted as a variable name and therefore the entire variable reference is replaced by the value of the referenced variable which is an empty string if the variable does not exist.

An example for a call of batch file unzip.cmd with the lines as posted in the question:

unzip.cmd "^#<!DATE!>%(!xyz!) & passphrase!$" "C:\Temp\Development & Test 100% (!).zip"

The output is:

Password after removing quotes is: #<24.07.2024>%() & passphrase$
Path: C:\Temp\Development & Test 100% ().zip

It can be seen how enabled delayed variable expansion caused a interpretation of the exclamation marks in the password and the file name strings with appropriate modification before command SET is executed at all.

1. The first character ^ of the password string is interpreted as escape character and for that reason removed from the password string.
2. !DATE! is interpreted as delayed expanded reference of the dynamic variable DATE resulting in replacing that string by the current date.
3. !xyz! is interpreted as delayed expanded reference of the not existing variable xyz resulting in this string being replaced by an empty string, i. e. removal of !xyz! from the password string.
4. The last ! in password string is just removed as there is no matching ! for interpreting the string between the exclamation marks as variable name.
5. The single exclamation mark in file name string is also removed for the same reason as the last exclamation mark in password string.

There must be nothing escaped on not enabling delayed variable expansion which is not needed at all on double quoting file/folder names and the password as always recommended in a batch file.

@echo off
setlocal EnableExtensions DisableDelayedExpansion
set "Password=%~1"
set "FileName=%~2"
setlocal EnableDelayedExpansion
echo Password:  !Password!
echo File Name: !FileName!
endlocal
"C:\Program Files\7-zip\7z.exe" x -o"%USERPROFILE%\Downloads" "-p%Password%" -y -- "%FileName%"
endlocal

Running this batch file with the same two arguments as before results in the output:

Password:  ^#<!DATE!>%(!xyz!) & passphrase!$
File Name: C:\Temp\Development & Test 100% (!).zip

There is no argument string modified anymore.

Delayed variable expansion is just enabled for the output of the two passed arguments strings without surrounding " in case of containing command operators like &, && or || or redirection operators like <, >, >> or >& which should be interpreted literally.

Important is that the password string is enclosed again in " on passing this string to 7-Zip whereby this executable supports enclosing the entire argument string with preceding -p in " or just the string itself as it is done with option -o. It is in general better enclosing the entire option in " as that is better for the parsing of the command line by cmd.exe.

The entire batch file can be simplified to:

@echo off
setlocal EnableExtensions DisableDelayedExpansion
"C:\Program Files\7-zip\7z.exe" x -o"%USERPROFILE%\Downloads" "-p%~1" -y -- "%~2"
endlocal

Command extensions are enabled by Windows default and delayed expansion are disabled by Windows default. The batch file can be reduced for that reason to the single line:

@"C:\Program Files\7-zip\7z.exe" x -o"%USERPROFILE%\Downloads" "-p%~1" -y -- "%~2"

A password string with " is not supported at all by cmd.exe.

A C# written program is not needed at all. If there is nevertheless used a C# coded program, there should be used the C# Process Class for execution of 7z.exe with the appropriate arguments and a batch file is no longer needed. The C# Process class is a C# wrapper class for the Windows kernel library function CreateProcess called without or with a STARTUPINFO structure.

If the fully qualified name of the Downloads folder is set as current working directory for 7z.exe on using the Process class, the option -o can be omitted on execution of 7-Zip. The Downloads folder is one of the known user shell folders as it can be seen on running in a command prompt window:

%SystemRoot%\System32\reg.exe QUERY "HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders"

respectively

%SystemRoot%\System32\reg.exe QUERY "HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders"

The Downloads folder has the registry value name {374DE290-123F-4565-9164-39C4925E467B}.

There are several C# solutions to get the Downloads folder path, see:

• How to programmatically derive Windows Downloads folder "%USERPROFILE%/Downloads"?
• Getting Downloads Folder in C#?

In a batch file can be used the following code to get the Downloads folder path as configured by the user.

set "DownloadsFolder="
for /F "skip=2 tokens=1,2*" %%I in ('%SystemRoot%\System32\reg.exe QUERY "HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\User Shell Folders" /v {374DE290-123F-4565-9164-39C4925E467B} 2^>nul') do if /I "%%I" == "{374DE290-123F-4565-9164-39C4925E467B}" if not "%%~K" == "" if "%%J" == "REG_SZ" (set "DownloadsFolder=%%~K") else if "%%J" == "REG_EXPAND_SZ" call set "DownloadsFolder=%%~K"
if not defined DownloadsFolder for /F "skip=2 tokens=1,2*" %%I in ('%SystemRoot%\System32\reg.exe QUERY "HKCU\Software\Microsoft\Windows\CurrentVersion\Explorer\Shell Folders" /v {374DE290-123F-4565-9164-39C4925E467B} 2^>nul') do if /I "%%I" == "{374DE290-123F-4565-9164-39C4925E467B}" if not "%%~K" == "" if "%%J" == "REG_SZ" (set "DownloadsFolder=%%~K") else if "%%J" == "REG_EXPAND_SZ" call set "DownloadsFolder=%%~K"
if not defined DownloadsFolder set "DownloadsFolder=\"
if "%DownloadsFolder:~-1%" == "\" set "DownloadsFolder=%DownloadsFolder:~0,-1%"
if not defined DownloadsFolder set "DownloadsFolder=%UserProfile%\Downloads"

Read my answer on path of user desktop in batch files for a detailed description of these six command lines.