#1 AI agent debug代码的一个完整案例
发表于 : 2025年 2月 25日 13:57
有三方通话:user, swe1, shell
我已经提前训练了AI使用shell以及我的定制化命令。训练时我作为user和swe1对话,我没有把训练内容贴出来, 如果贴出来的话就是前面还有大概这么多邮件交互。下面的记录是全自动跑的, user和swe1背后都是gpt4o。因为训练时的user是我,所以你们看到下面的user其实算是gpt4o模仿我的口气给swe1发邮件。凡是调API时会有一个M-Cost邮件头。
这是swe-bench里最简单的案例之一。但这确实是django在github上的真实案例,以前这种都是要人fix的。
阅读方法是注意看看----分界下面的From:, To:.
AI调用shell的协议是: Subject:的内容是命令,如果邮件有body,则body的内容作为stdin。AI大致的思路是先用grep找对应的程序,找到后用aa_list显示程序内容,然后用aa_edit修改。最后用aa_test测试。测试通过我的driver会自动终止。如果测试通不过,AI会接着试别的办法,这时候往往就跑飞了. Driver会track花了多少钱,到了预定的budget就会终止循环。 像这么一个session差不多花一两毛钱。 复杂点的问题一块钱很快就花掉了。
From --------------------------------
From: user@localdomain
To: swe1@localdomain
Subject: New project
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
We are now in a new codebase. Solve the issue as we did before.
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: aa_open ./.swe/problem_statement.txt
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 3196
M-Tokens-Output: 35
M-Cost: 0.00834000
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
--- stdout ---
current file: ./.swe/problem_statement.txt
total lines: 9
cannot scroll
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: aa_list 1-9
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 3290
M-Tokens-Output: 32
M-Cost: 0.00854500
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
--- stdout ---
1|UsernameValidator allows trailing newline in usernames
2|Description
3|
4|ASCIIUsernameValidator and UnicodeUsernameValidator use the regex
5|r'^[\w.@+-]+$'
6|The intent is to only allow alphanumeric characters as well as ., @, +, =
and -. However, a little known quirk of Python regexes is that $ will also ma=
tch a trailing newline. Therefore, the user name validators will accept usern=
ames which end with a newline. You can avoid this behavior by instead using \=
A and \Z to terminate regexes. For example, the validator regex could be chan=
ged to
7|r'\A[\w.@+-]+\Z'
8|in order to reject usernames that end with a newline.
9|I am not sure how to officially post a patch, but the required change is=
trivial - using the regex above in the two validators in contrib.auth.valida=
tors.
current file: ./.swe/problem_statement.txt
total lines: 9
can scroll
From --------------------------------
From: swe1@localdomain
To: user@localdomain
Subject: Approach to Fix the Username Validator Issue
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 3582
M-Tokens-Output: 225
M-Cost: 0.01120500
The issue at hand involves the `UsernameValidator` in Django which incorrectly allows trailing newlines in usernames due to the use of the `^` and `$` anchors in its regular expressions. We need to modify the regex pattern to use `\A` and `\Z` instead, to ensure trailing newlines are not allowed.
**Approach to Fix the Issue:**
1. **Identify the Validators:**
- Locate `ASCIIUsernameValidator` and `UnicodeUsernameValidator` implementations within the codebase.
2. **Update Regex Pattern:**
- Change the regex pattern from `r'^[\w.@+-]+$'` to `r'\A[\w.@+-]+\Z'` to prevent trailing newlines.
3. **Testing:**
- Run any relevant tests to ensure the changes work correctly and trailing newlines are no longer permissible.
Let's proceed by locating these validators in the codebase and making the necessary changes.
From --------------------------------
From: user@localdomain
To: swe1@localdomain
Subject:
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 1687
M-Tokens-Output: 34
M-Cost: 0.00455750
Locate the relevant file to apply the changes.
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: grep -r "UsernameValidator" ./django/contrib/auth/validators
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 3847
M-Tokens-Output: 39
M-Cost: 0.01000750
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 2
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
--- stderr ---
grep: ./django/contrib/auth/validators: No such file or directory
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: grep -r "UsernameValidator" ./django/contrib/auth/
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 3941
M-Tokens-Output: 39
M-Cost: 0.01024250
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
--- stdout ---
./django/contrib/auth/models.py:from .validators import UnicodeUsernameValida=
tor
./django/contrib/auth/models.py: username_validator =3D UnicodeUsernameVal=
idator()
./django/contrib/auth/migrations/0007_alter_validators_add_error_messages.py:=
validators=3D[validators.UnicodeUsernameValidator()],
./django/contrib/auth/migrations/0008_alter_user_username_max_length.py: =
validators=3D[validators.UnicodeUsernameValidator()],
./django/contrib/auth/migrations/0004_alter_user_username_opts.py: =
validators=3D[validators.UnicodeUsernameValidator()],
./django/contrib/auth/migrations/0001_initial.py: validato=
rs=3D[validators.UnicodeUsernameValidator()],
./django/contrib/auth/validators.py:class ASCIIUsernameValidator(validators.R=
egexValidator):
./django/contrib/auth/validators.py:class UnicodeUsernameValidator(validators=
.RegexValidator):
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: aa_open ./django/contrib/auth/validators.py
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 4190
M-Tokens-Output: 36
M-Cost: 0.01083500
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
--- stdout ---
current file: ./django/contrib/auth/validators.py
total lines: 25
cannot scroll
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: aa_list 1-25
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 4286
M-Tokens-Output: 32
M-Cost: 0.01103500
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
--- stdout ---
1|import re
2|
3|from django.core import validators
4|from django.utils.deconstruct import deconstructible
5|from django.utils.translation import gettext_lazy as _
6|
7|
8|@deconstructible
9|class ASCIIUsernameValidator(validators.RegexValidator):
10| regex =3D r'^[\w.@+-]+$'
11| message =3D _(
12| 'Enter a valid username. This value may contain only English let=
ters, '
13| 'numbers, and @/./+/-/_ characters.'
14| )
15| flags =3D re.ASCII
16|
17|
18|@deconstructible
19|class UnicodeUsernameValidator(validators.RegexValidator):
20| regex =3D r'^[\w.@+-]+$'
21| message =3D _(
22| 'Enter a valid username. This value may contain only letters, '
23| 'numbers, and @/./+/-/_ characters.'
24| )
25| flags =3D 0
current file: ./django/contrib/auth/validators.py
total lines: 25
can scroll
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: aa_edit 10-20
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 4642
M-Tokens-Output: 121
M-Cost: 0.01281500
regex = r'\A[\w.@+-]+\Z'
message = _(
'Enter a valid username. This value may contain only English letters, '
'numbers, and @/./+/-/_ characters.'
)
flags = re.ASCII
@deconstructible
class UnicodeUsernameValidator(validators.RegexValidator):
regex = r'\A[\w.@+-]+\Z'
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
--- stdout ---
11 lines replaced.
current file: ./django/contrib/auth/validators.py
total lines: 25
cannot scroll
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: aa_test
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 4827
M-Tokens-Output: 28
M-Cost: 0.01234750
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
--- stdout ---
0c5641a4f08f798f24001d2c1bd3bc9e5d070a873250a80be8d158a5281e4772
Congratulations! You have passed the test.
我已经提前训练了AI使用shell以及我的定制化命令。训练时我作为user和swe1对话,我没有把训练内容贴出来, 如果贴出来的话就是前面还有大概这么多邮件交互。下面的记录是全自动跑的, user和swe1背后都是gpt4o。因为训练时的user是我,所以你们看到下面的user其实算是gpt4o模仿我的口气给swe1发邮件。凡是调API时会有一个M-Cost邮件头。
这是swe-bench里最简单的案例之一。但这确实是django在github上的真实案例,以前这种都是要人fix的。
阅读方法是注意看看----分界下面的From:, To:.
AI调用shell的协议是: Subject:的内容是命令,如果邮件有body,则body的内容作为stdin。AI大致的思路是先用grep找对应的程序,找到后用aa_list显示程序内容,然后用aa_edit修改。最后用aa_test测试。测试通过我的driver会自动终止。如果测试通不过,AI会接着试别的办法,这时候往往就跑飞了. Driver会track花了多少钱,到了预定的budget就会终止循环。 像这么一个session差不多花一两毛钱。 复杂点的问题一块钱很快就花掉了。
From --------------------------------
From: user@localdomain
To: swe1@localdomain
Subject: New project
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
We are now in a new codebase. Solve the issue as we did before.
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: aa_open ./.swe/problem_statement.txt
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 3196
M-Tokens-Output: 35
M-Cost: 0.00834000
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
--- stdout ---
current file: ./.swe/problem_statement.txt
total lines: 9
cannot scroll
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: aa_list 1-9
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 3290
M-Tokens-Output: 32
M-Cost: 0.00854500
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
--- stdout ---
1|UsernameValidator allows trailing newline in usernames
2|Description
3|
4|ASCIIUsernameValidator and UnicodeUsernameValidator use the regex
5|r'^[\w.@+-]+$'
6|The intent is to only allow alphanumeric characters as well as ., @, +, =
and -. However, a little known quirk of Python regexes is that $ will also ma=
tch a trailing newline. Therefore, the user name validators will accept usern=
ames which end with a newline. You can avoid this behavior by instead using \=
A and \Z to terminate regexes. For example, the validator regex could be chan=
ged to
7|r'\A[\w.@+-]+\Z'
8|in order to reject usernames that end with a newline.
9|I am not sure how to officially post a patch, but the required change is=
trivial - using the regex above in the two validators in contrib.auth.valida=
tors.
current file: ./.swe/problem_statement.txt
total lines: 9
can scroll
From --------------------------------
From: swe1@localdomain
To: user@localdomain
Subject: Approach to Fix the Username Validator Issue
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 3582
M-Tokens-Output: 225
M-Cost: 0.01120500
The issue at hand involves the `UsernameValidator` in Django which incorrectly allows trailing newlines in usernames due to the use of the `^` and `$` anchors in its regular expressions. We need to modify the regex pattern to use `\A` and `\Z` instead, to ensure trailing newlines are not allowed.
**Approach to Fix the Issue:**
1. **Identify the Validators:**
- Locate `ASCIIUsernameValidator` and `UnicodeUsernameValidator` implementations within the codebase.
2. **Update Regex Pattern:**
- Change the regex pattern from `r'^[\w.@+-]+$'` to `r'\A[\w.@+-]+\Z'` to prevent trailing newlines.
3. **Testing:**
- Run any relevant tests to ensure the changes work correctly and trailing newlines are no longer permissible.
Let's proceed by locating these validators in the codebase and making the necessary changes.
From --------------------------------
From: user@localdomain
To: swe1@localdomain
Subject:
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 1687
M-Tokens-Output: 34
M-Cost: 0.00455750
Locate the relevant file to apply the changes.
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: grep -r "UsernameValidator" ./django/contrib/auth/validators
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 3847
M-Tokens-Output: 39
M-Cost: 0.01000750
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 2
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
--- stderr ---
grep: ./django/contrib/auth/validators: No such file or directory
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: grep -r "UsernameValidator" ./django/contrib/auth/
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 3941
M-Tokens-Output: 39
M-Cost: 0.01024250
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
--- stdout ---
./django/contrib/auth/models.py:from .validators import UnicodeUsernameValida=
tor
./django/contrib/auth/models.py: username_validator =3D UnicodeUsernameVal=
idator()
./django/contrib/auth/migrations/0007_alter_validators_add_error_messages.py:=
validators=3D[validators.UnicodeUsernameValidator()],
./django/contrib/auth/migrations/0008_alter_user_username_max_length.py: =
validators=3D[validators.UnicodeUsernameValidator()],
./django/contrib/auth/migrations/0004_alter_user_username_opts.py: =
validators=3D[validators.UnicodeUsernameValidator()],
./django/contrib/auth/migrations/0001_initial.py: validato=
rs=3D[validators.UnicodeUsernameValidator()],
./django/contrib/auth/validators.py:class ASCIIUsernameValidator(validators.R=
egexValidator):
./django/contrib/auth/validators.py:class UnicodeUsernameValidator(validators=
.RegexValidator):
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: aa_open ./django/contrib/auth/validators.py
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 4190
M-Tokens-Output: 36
M-Cost: 0.01083500
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
--- stdout ---
current file: ./django/contrib/auth/validators.py
total lines: 25
cannot scroll
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: aa_list 1-25
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 4286
M-Tokens-Output: 32
M-Cost: 0.01103500
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
--- stdout ---
1|import re
2|
3|from django.core import validators
4|from django.utils.deconstruct import deconstructible
5|from django.utils.translation import gettext_lazy as _
6|
7|
8|@deconstructible
9|class ASCIIUsernameValidator(validators.RegexValidator):
10| regex =3D r'^[\w.@+-]+$'
11| message =3D _(
12| 'Enter a valid username. This value may contain only English let=
ters, '
13| 'numbers, and @/./+/-/_ characters.'
14| )
15| flags =3D re.ASCII
16|
17|
18|@deconstructible
19|class UnicodeUsernameValidator(validators.RegexValidator):
20| regex =3D r'^[\w.@+-]+$'
21| message =3D _(
22| 'Enter a valid username. This value may contain only letters, '
23| 'numbers, and @/./+/-/_ characters.'
24| )
25| flags =3D 0
current file: ./django/contrib/auth/validators.py
total lines: 25
can scroll
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: aa_edit 10-20
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 4642
M-Tokens-Output: 121
M-Cost: 0.01281500
regex = r'\A[\w.@+-]+\Z'
message = _(
'Enter a valid username. This value may contain only English letters, '
'numbers, and @/./+/-/_ characters.'
)
flags = re.ASCII
@deconstructible
class UnicodeUsernameValidator(validators.RegexValidator):
regex = r'\A[\w.@+-]+\Z'
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
--- stdout ---
11 lines replaced.
current file: ./django/contrib/auth/validators.py
total lines: 25
cannot scroll
From --------------------------------
From: swe1@localdomain
To: shell@localdomain
Subject: aa_test
Content-Type: text/plain
M-Model: openai/gpt-4o
M-Tokens-Input: 4827
M-Tokens-Output: 28
M-Cost: 0.01234750
From --------------------------------
From: shell@localdomain
To: swe1@localdomain
Subject: Exit Code: 0
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: 7bit
MIME-Version: 1.0
--- stdout ---
0c5641a4f08f798f24001d2c1bd3bc9e5d070a873250a80be8d158a5281e4772
Congratulations! You have passed the test.