glibc提供的malloc()的调试工具
2024-09-06 08:26:31
关键词：MALLOC_CHECK_、mtrace()、muntrace()、MALLOC_TRACE、mprobe()、-lmcheck等等。
1. MALLOC_CHECK_环境变量(double free)
MALLOC_CHECK_提供了类似于mcheck()和mprobe()函数的功能，但是无需对程序进行修改和重新编译。
设置不同整数值可以控制程序对内存分配错误的响应方式。
0 - 不产生错误信息，也不中止这个程序
1 - 产生错误信息，但是不中止这个程序
2 - 不产生错误信息，但是中止这个程序
3 - 产生错误信息，并中止这个程序
下面构造一个double free的错误程序验证一下：
#include <stdio.h>
#include <malloc.h> void main(void)
{
char *s = NULL; s = malloc();
free(s);
free(s);
}
env MALLOC_CHECK_=0 ./dfree的效果类似于直接执行./dfree。
env MALLOC_CHECK_=1 ./dfree产生如下的错误信息：
*** Error in `./dfree': free(): invalid pointer: 0x0000000000e22010 ***
env MALLOC_CHECK_=2 ./dfree简单的终止程序，生成coredump文件。
Aborted (core dumped)
通过分析core文件，然后bt full查看backtrace：
(gdb) bt full
# 0x00007fb4cd69c428 in __GI_raise (sig=sig@entry=) at ../sysdeps/unix/sysv/linux/raise.c:
resultvar =
pid =
selftid =
# 0x00007fb4cd69e02a in __GI_abort () at abort.c:
save_stage =
act = {__sigaction_handler = {sa_handler = 0x0, sa_sigaction = 0x0}, sa_mask = {__val = { <repeats times>, , }}, sa_flags = , sa_restorer = 0x0}
sigs = {__val = {, <repeats times>}}
# 0x00007fb4cd6e6430 in malloc_printerr (ar_ptr=<optimized out>, ptr=<optimized out>, str=<optimized out>, action=<optimized out>) at malloc.c:
No locals.
# free_check (mem=<optimized out>, caller=<optimized out>) at hooks.c:
No locals.
# 0x00007fb4cd6eb5e7 in __GI___libc_free (mem=<optimized out>) at malloc.c:
ar_ptr = <optimized out>
p = <optimized out>
hook = <optimized out>
# 0x000000000040059c in main ()
No symbol table info available.
env MALLOC_CHECK_=3 ./dfree显示更多的信息并且coredump。
*** Error in `./dfree': free(): invalid pointer: 0x0000000000aa0010 ***
======= Backtrace: =========--------------------------------------------------------------------问题点的backtrace。
/lib/x86_64-linux-gnu/libc.so.(+0x777e5)[0x7f640d3447e5]
/lib/x86_64-linux-gnu/libc.so.(+0x7f72a)[0x7f640d34c72a]
/lib/x86_64-linux-gnu/libc.so.(cfree+0xf7)[0x7f640d3515e7]
./dfree[0x40059c]
/lib/x86_64-linux-gnu/libc.so.(__libc_start_main+0xf0)[0x7f640d2ed830]
./dfree[0x400499]
======= Memory map: ========---------------------------------------------------------------------当前进程的maps。
- r-xp : /home/al/test/dfree
- r--p : /home/al/test/dfree
- rw-p : /home/al/test/dfree
00aa0000-00ac1000 rw-p : [heap]
7f640d0b7000-7f640d0cd000 r-xp : /lib/x86_64-linux-gnu/libgcc_s.so.
7f640d0cd000-7f640d2cc000 ---p : /lib/x86_64-linux-gnu/libgcc_s.so.
7f640d2cc000-7f640d2cd000 rw-p : /lib/x86_64-linux-gnu/libgcc_s.so.
7f640d2cd000-7f640d48d000 r-xp : /lib/x86_64-linux-gnu/libc-2.23.so
7f640d48d000-7f640d68d000 ---p 001c0000 : /lib/x86_64-linux-gnu/libc-2.23.so
7f640d68d000-7f640d691000 r--p 001c0000 : /lib/x86_64-linux-gnu/libc-2.23.so
7f640d691000-7f640d693000 rw-p 001c4000 : /lib/x86_64-linux-gnu/libc-2.23.so
7f640d693000-7f640d697000 rw-p :
7f640d697000-7f640d6bd000 r-xp : /lib/x86_64-linux-gnu/ld-2.23.so
7f640d890000-7f640d893000 rw-p :
7f640d8bb000-7f640d8bc000 rw-p :
7f640d8bc000-7f640d8bd000 r--p : /lib/x86_64-linux-gnu/ld-2.23.so
7f640d8bd000-7f640d8be000 rw-p : /lib/x86_64-linux-gnu/ld-2.23.so
7f640d8be000-7f640d8bf000 rw-p :
7ffc0fb27000-7ffc0fb49000 rw-p : [stack]
7ffc0fbb2000-7ffc0fbb5000 r--p : [vvar]
7ffc0fbb5000-7ffc0fbb7000 r-xp : [vdso]
ffffffffff600000-ffffffffff601000 r-xp : [vsyscall]
Aborted (core dumped)
下面是详细的栈信息：
# 0x00007f640d302428 in __GI_raise (sig=sig@entry=) at ../sysdeps/unix/sysv/linux/raise.c:
resultvar =
pid =
selftid =
# 0x00007f640d30402a in __GI_abort () at abort.c:
save_stage =
act = {__sigaction_handler = {sa_handler = 0x2070782d72203030, sa_sigaction = 0x2070782d72203030}, sa_mask = {__val = {, , , ,
, , , , , , , , , ,
, }}, sa_flags = , sa_restorer = 0x48}
sigs = {__val = {, <repeats times>}}
# 0x00007f640d3447ea in __libc_message (do_abort=do_abort@entry=, fmt=fmt@entry=0x7f640d45ded8 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:
ap = <error reading variable ap (Attempt to dereference a generic pointer.)>
fd =
on_2 = <optimized out>
list = <optimized out>
nlist = <optimized out>
cp = <optimized out>
written = <optimized out>
# 0x00007f640d34c72a in malloc_printerr (ar_ptr=0x7f640d691b20 <main_arena>, ptr=<optimized out>, str=0x7f640d45acaf "free(): invalid pointer", action=<optimized out>) at malloc.c:
buf = "0000000000aa0010"
cp = <optimized out>
ar_ptr = 0x7f640d691b20 <main_arena>
ptr = <optimized out>
str = 0x7f640d45acaf "free(): invalid pointer"
action = <optimized out>
# free_check (mem=<optimized out>, caller=<optimized out>) at hooks.c:
No locals.
# 0x00007f640d3515e7 in __GI___libc_free (mem=<optimized out>) at malloc.c:
ar_ptr = <optimized out>
p = <optimized out>
hook = <optimized out>
# 0x000000000040059c in main ()
No symbol table info available.
2. mtrace/muntrace/MALLOC_TRACE(重复释放、泄漏)
mtrace()和muntrace()函数分别在程序中打开和关闭对内存分配调用进行跟踪的功能。
这两个函数要与环境变量MALLOC_TRACE搭配使用，该变量定义了写入跟踪信息的文件名。
2.1 构造测试用例
构造malloc()但是不释放的场景：
#include <stdlib.h>
#include <stdio.h>
#include <mcheck.h> int main(int argc, char **argv)
{
mtrace(); char * p = malloc();
free(p); p = malloc(); muntrace(); return ;
}
2.2 mtrace()跟踪结果分析
gcc mtrace.c -o mtrace -g编译带调试信息。
export MALLOC_TRACE=/home/al/test/mtrace.log设置mtrace()信息输出路径。
然后执行./mtrace，就会在MALLOC_TRACE下生成malloc()/free()轨迹信息。
= Start
@ ./mtrace:[0x400624] + 0x10b4450 0x64
@ ./mtrace:[0x400634] - 0x10b4450
@ ./mtrace:[0x40063e] + 0x10b44c0 0x3e8
= End
上面的信息格式为：@ 程序名称:[内存分配释放调用的地址] +/- 操作的内存地址 参数。
+表示malloc，-表示free。
或者如下类型log：
@ 程序名称:(函数名称+偏移量) [内存分配释放调用地址] +/-/</> 操作内存地址 参数。
+：对应一个malloc()操作。
-：对应一个free()操作。
</>：对应一个realloc()操作，成对出现。<表示释放之前内存，>表示申请后结果。如果realloc()的内存第一次申请，那么就对应一个+号。
@ /usr/lib/libglib-2.0.so.:(g_malloc+0x2a)[0x2ac7c412] + 0x2b510f50 0x3------------------------g_malloc()中申请0x3字节大小内存，返回地址为0x2b510f50。
@ /usr/lib/libglib-2.0.so.:(g_realloc+0x38)[0x2ac7c4cc] + 0x2b510f60 0x20----------------------g_realloc()指针之前没有对应内存，所以对应+。表示申请0x20字节大小内存，返回地址为0x2b510f60。
@ /usr/lib/libglib-2.0.so.:(g_free+0x20)[0x2ac7c524] - 0x2b510f40
@ /usr/lib/libglib-2.0.so.:(g_malloc+0x2a)[0x2ac7c412] + 0x2b510f40 0xc------------------------在g_malloc()中申请0xc字节大小内存，返回地址为0x2b510f40.
@ /usr/lib/libglib-2.0.so.:(g_malloc+0x2a)[0x2ac7c412] + 0x2b510f88 0xc
@ /usr/lib/libglib-2.0.so.:(g_realloc+0x38)[0x2ac7c4cc] < 0x2b510f60---------------------------g_realloc()对应的内存已经存在，所以先释放0x2b510f60内存。
@ /usr/lib/libglib-2.0.so.:(g_realloc+0x38)[0x2ac7c4cc] > 0x2b510f98 0x40----------------------然后重新申请0x40大小的内存，返回地址为0x2b50f98.
@ /usr/lib/libglib-2.0.so.:(g_free+0x20)[0x2ac7c524] - 0x2b510f40------------------------------在g_free()中释放地址为0x2b510f40的内存。
@ /usr/lib/libglib-2.0.so.:(g_realloc+0x38)[0x2ac7c4cc] < 0x2b510f98---------------------------g_realoc()对应内存已经存在，释放0x2b510f98对应内存。
@ /usr/lib/libglib-2.0.so.:(g_realloc+0x38)[0x2ac7c4cc] > 0x2b513660 0x80----------------------然后重新申请0x80大小内存，返回地址为0x2b513660。
通过mtrace mtrace mtrace.log可以可读性更强的信息。第一个mtrace是解析mtrace.log的工具，第二个mtrace是测试程序。
Memory not freed:
-----------------
Address Size Caller
0x00000000010b44c0 0x3e8 at /home/al/test/mtrace.c:
这里面对过滤掉正常malloc()/free()信息，留下的是泄漏内存。
Address表示泄漏地址，Size表示泄漏大小，Caller表示泄漏点代码位置。
2.3 重复释放问题
如下是重复释放问题，mtrace()会如何处理呢？
#include <stdio.h>
#include <malloc.h>
#include <mcheck.h> void main(void)
{
char *s = NULL;
mtrace();
s = malloc();
free(s);
free(s);
muntrace();
}
执行./dfree结果如下：
*** Error in `./dfree': double free or corruption (fasttop): 0x000000000188b450 ***
2.4 mtrace输出解释
mtrace输出结果还有一些可能不是异常的提示。
= Start
[0x8048209] - 0x8064cc8
[0x8048209] - 0x8064ce0
[0x8048209] - 0x8064cf8
[0x80481eb] + 0x8064c48 0x14
[0x80481eb] + 0x8064c60 0x14
[0x80481eb] + 0x8064c78 0x14
[0x80481eb] + 0x8064c90 0x14
= End
输出结果如下：
- 0x08064cc8 Free was never alloc'd /home/drepper/tst.c:39
- 0x08064ce0 Free was never alloc'd /home/drepper/tst.c:39
- 0x08064cf8 Free was never alloc'd /home/drepper/tst.c:39
(-表示free异常) (free内存对应指针) Free (在原log中的行号) was never alloc'd (free函数调用点) Memory not freed:
-----------------
Address Size Caller
0x08064c48 0x14 at /home/drepper/tst.c:
0x08064c60 0x14 at /home/drepper/tst.c:
0x08064c78 0x14 at /home/drepper/tst.c:
0x08064c90 0x14 at /home/drepper/tst.c:
参考资料：《3.2.4.4 Interpreting the traces》。
- 0x000000000002ba00 Realloc was never alloc'd 0x2af654bc
(-表示释放) (free对应内存地址) Realloc (在log中行号) was never alloc'd (free释放点)
realloc()首先释放然后重新申请内存，表示在realloc()释放内存的时候没有找到对应的alloc()。
+ 0x000000000006ea48 Alloc duplicate: 0x2b2ef800 /lib/libstdc++.so.:(_Znwj+0x24)[0x2b2ef800]
(+表示alloc()) (alloc()内存地址) Alloc (在log中行号) duplicate: (alloc()调用点)
表示两次alloc两次，申请的内存地址是一样的。前一次alloc()到本次alloc()之间没有free。
3. mcheck()检查内存一致性(重复释放、越界)
mcheck()函数允许程序对已分配内存块进行一致性检查。
3.1 mcheck()介绍
#include <mcheck.h> int mcheck(void (*abortfunc)(enum mcheck_status mstatus));
调用该函数后，后续内存分配、释放都将进行内存连续性检查，并在内存连续性检查失败后，调用abortfunc。
枚举体mcheck_status如下：
enum mcheck_status
{
MCHECK_DISABLED = -, /* Consistency checking is not turned on. */
MCHECK_OK, /* Block is fine. */
MCHECK_FREE, /* Block freed twice. */
MCHECK_HEAD, /* Memory before the block was clobbered. */
MCHECK_TAIL /* Memory after the block was clobbered. */
};
3.2 mcheck()实例
下例创建捕获错误函数abortfun()，三种你错误类型：重复释放、头覆盖、尾覆盖。
#include <stdio.h>
#include <malloc.h>
#include <mcheck.h>
#include <errno.h>
#include <string.h> void abortfun(enum mcheck_status mstatus)
{
if(mstatus == MCHECK_FREE)
fprintf(stderr, "Block freed twice.\n");
else if(mstatus == MCHECK_HEAD)
fprintf(stderr, "Memory before the block was clobbered.\n");
else if(mstatus == MCHECK_TAIL)
fprintf(stderr, "Memory after the block was clobbered.\n");
else
fprintf(stderr, "Block is fine.\n");
} void main(void)
{
char *s = NULL; if(mcheck(abortfun) != )
{
fprintf(stderr, "mcheck:%s\n", strerror(errno));
return;
}
s = malloc();
*(s-) = ;--------------------------头覆盖。
*(s+) = ;-------------------------尾覆盖。
free(s);
free(s);-----------------------------重复释放。
}
执行后结果如下，捕获到了尾覆盖和重复释放，头覆盖没有捕获到；但是单独头覆盖是可以捕获到的。
并且产生了coredump文件。
Memory after the block was clobbered.
Block freed twice.
*** Error in `./dfree': double free or corruption (fasttop): 0x0000000000957030 ***
======= Backtrace: =========
/lib/x86_64-linux-gnu/libc.so.(+0x777e5)[0x7fc2eac267e5]
/lib/x86_64-linux-gnu/libc.so.(+0x8037a)[0x7fc2eac2f37a]
/lib/x86_64-linux-gnu/libc.so.(cfree+0x4c)[0x7fc2eac3353c]
/lib/x86_64-linux-gnu/libc.so.(+0x87fa0)[0x7fc2eac36fa0]
/lib/x86_64-linux-gnu/libc.so.(cfree+0xf7)[0x7fc2eac335e7]
./dfree[0x40084a]
/lib/x86_64-linux-gnu/libc.so.(__libc_start_main+0xf0)[0x7fc2eabcf830]
./dfree[0x400659]
======= Memory map: ========
- r-xp : /home/al/test/dfree
- r--p : /home/al/test/dfree
- rw-p : /home/al/test/dfree
- rw-p : [heap]
7fc2e4000000-7fc2e4021000 rw-p :
7fc2e4021000-7fc2e8000000 ---p :
7fc2ea999000-7fc2ea9af000 r-xp : /lib/x86_64-linux-gnu/libgcc_s.so.
7fc2ea9af000-7fc2eabae000 ---p : /lib/x86_64-linux-gnu/libgcc_s.so.
7fc2eabae000-7fc2eabaf000 rw-p : /lib/x86_64-linux-gnu/libgcc_s.so.
7fc2eabaf000-7fc2ead6f000 r-xp : /lib/x86_64-linux-gnu/libc-2.23.so
7fc2ead6f000-7fc2eaf6f000 ---p 001c0000 : /lib/x86_64-linux-gnu/libc-2.23.so
7fc2eaf6f000-7fc2eaf73000 r--p 001c0000 : /lib/x86_64-linux-gnu/libc-2.23.so
7fc2eaf73000-7fc2eaf75000 rw-p 001c4000 : /lib/x86_64-linux-gnu/libc-2.23.so
7fc2eaf75000-7fc2eaf79000 rw-p :
7fc2eaf79000-7fc2eaf9f000 r-xp : /lib/x86_64-linux-gnu/ld-2.23.so
7fc2eb172000-7fc2eb175000 rw-p :
7fc2eb19d000-7fc2eb19e000 rw-p :
7fc2eb19e000-7fc2eb19f000 r--p : /lib/x86_64-linux-gnu/ld-2.23.so
7fc2eb19f000-7fc2eb1a0000 rw-p : /lib/x86_64-linux-gnu/ld-2.23.so
7fc2eb1a0000-7fc2eb1a1000 rw-p :
7ffcc5840000-7ffcc5862000 rw-p : [stack]
7ffcc58df000-7ffcc58e2000 r--p : [vvar]
7ffcc58e2000-7ffcc58e4000 r-xp : [vdso]
ffffffffff600000-ffffffffff601000 r-xp : [vsyscall]
Aborted (core dumped)
4. mprobe()
函数格式介绍：
#include <mcheck.h> enum mcheck_status mprobe(void *ptr);
mprobe()示例：
#include <stdio.h>
#include <malloc.h>
#include <mcheck.h>
#include <errno.h>
#include <string.h> void abortfun(enum mcheck_status mstatus)
{
if(mstatus == MCHECK_FREE)
fprintf(stderr, "Block freed twice.\n");
else if(mstatus == MCHECK_HEAD)
fprintf(stderr, "Memory before the block was clobbered.\n");
else if(mstatus == MCHECK_TAIL)
fprintf(stderr, "Memory after the block was clobbered.\n");
else
fprintf(stderr, "Block is fine.\n");
} void main(void)
{
char *s = NULL; if(mcheck(abortfun) != )
{
fprintf(stderr, "mcheck:%s\n", strerror(errno));
return;
}
s = malloc();
mprobe(s);------------------------------正确
mprobe(s-);----------------------------错误，返回MCHECK_HEAD错误类型。
mprobe(s+);---------------------------错误，返回MCHECK_HEAD错误类型。
free(s);
}
返回结果：
Memory before the block was clobbered.
Memory before the block was clobbered.
5. -lmcheck自动开启检查
在编译的时候加上-lmcheck，不需要修改代码就可以对malloc()/free()进行检查。
#include <stdio.h>
#include <malloc.h> void main(void)
{
char *s = NULL; s = malloc();
free(s);
free(s);
}
gcc dfree.c -o dfree -lmcheck编译后，执行./dfree。
block freed twice
Aborted (core dumped)
查看coredump bt full如下：
# 0x00007fdf1e826428 in __GI_raise (sig=sig@entry=) at ../sysdeps/unix/sysv/linux/raise.c:
resultvar =
pid =
selftid =
# 0x00007fdf1e82802a in __GI_abort () at abort.c:
save_stage =
act = {__sigaction_handler = {sa_handler = 0x3065383363666637, sa_sigaction = 0x3065383363666637}, sa_mask = {__val = {, , , ,
, , , , , , , , , ,
, }}, sa_flags = , sa_restorer = 0x54}
sigs = {__val = {, <repeats times>}}
# 0x00007fdf1e8687ea in __libc_message (do_abort=do_abort@entry=, fmt=fmt@entry=0x7fdf1e981ed8 "*** Error in `%s': %s: 0x%s ***\n") at ../sysdeps/posix/libc_fatal.c:
ap = <error reading variable ap (Attempt to dereference a generic pointer.)>
fd =
on_2 = <optimized out>
list = <optimized out>
nlist = <optimized out>
cp = <optimized out>
written = <optimized out>
# 0x00007fdf1e87137a in malloc_printerr (ar_ptr=<optimized out>, ptr=<optimized out>, str=0x7fdf1e981fa0 "double free or corruption (fasttop)", action=) at malloc.c:
buf = "000000000186a010"
cp = <optimized out>
ar_ptr = <optimized out>
str = 0x7fdf1e981fa0 "double free or corruption (fasttop)"
action =
# _int_free (av=<optimized out>, p=<optimized out>, have_lock=) at malloc.c:
size = <optimized out>
fb = <optimized out>
nextchunk = <optimized out>
nextsize = <optimized out>
nextinuse = <optimized out>
prevsize = <optimized out>
bck = <optimized out>
fwd = <optimized out>
errstr = <optimized out>
locked = <optimized out>
# 0x00007fdf1e87553c in __GI___libc_free (mem=<optimized out>) at malloc.c:
ar_ptr = <optimized out>
p = <optimized out>
hook = <optimized out>
# 0x000000000040059c in register_tm_clones ()
No symbol table info available.
# 0x00007ffc38dc54c0 in ?? ()
No symbol table info available.
# 0x000000000186a010 in ?? ()
No symbol table info available.
# 0x00000000004005a0 in register_tm_clones ()
No symbol table info available.
# 0x00007fdf1e811830 in __libc_start_main (main=0x400566 <deregister_tm_clones+>, argc=, argv=0x7ffc38dc54c8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>, stack_end=0x7ffc38dc54b8)
at ../csu/libc-start.c:
result = <optimized out>
unwind_buf = {cancel_jmp_buf = {{jmp_buf = {, , , , , , -, -}, mask_was_saved = }}, priv = {pad = {0x0, 0x0, 0x400610 <main+>,
0x7fdf1ebcbab0 <_dl_fini>}, data = {prev = 0x0, cleanup = 0x0, canceltype = }}}
not_first_call = <optimized out>
# 0x0000000000400499 in ?? ()
No symbol table info available.
# 0x00007ffc38dc54b8 in ?? ()
No symbol table info available.
# 0x000000000000001c in ?? ()
No symbol table info available.
# 0x0000000000000001 in ?? ()
No symbol table info available.
# 0x00007ffc38dc5fba in ?? ()
No symbol table info available.
# 0x0000000000000000 in ?? ()
No symbol table info available.
6. 小结
对比以上几个内存你检查手段：MALLOC_CHECK_最简单，其次是-lmcheck，最后是mtrace()/muntrace()、mcheck()、mprobe()。
但是这几种技术检查的全面性都不够，没有一种能够全面检查内存泄漏、内存踩踏、重复释放的。
要想全面的检查还是需要Valgrind这种技术，参考《valgrind使用方法》。
参考文档：《mtrace-内存使用追踪（内存）》、《mcheck 函数使用（glibc-3-内存）》
最新文章
- !+"\v1" 用来“判断浏览器类型”还是用来“IE判断版本”的问题！
- git查看日志
- UVa 11426 - GCD - Extreme (II)
- （7）nehe教程1 创建一个OpenGL窗口:
- ios新特性
- GET和POST本质上有什么区别
- C++随机崩溃捕捉处理
- 一统江湖的大前端（6）commander.js + inquirer.js——懒，才是第一生产力
- luogu5010 HMR的LIS III (dp+线段树)
- centos7.4/rehat7.0系统安装
- POJ 2909
- Android：XML简介 & 解析方式对比（DOM、SAX、PULL）
- C# HtmlDocument和HtmlNode的使用以及节点的模糊查询
- Spark项目之电商用户行为分析大数据平台之（二）CentOS7集群搭建
- Java重写toString和泛型的使用
- Android动画知识汇总
- webp图片优化
- 关于类属性值校验的一点记录 【知识点Attribute】
- web框架引入
- java 记录
热门文章
- 【zabbix监控】zabbix监控tomcat服务
- Linux下使用docker 拉取 vsftpd 镜像搭建 Ftp 服务器，连接 Ftp 时遇到的错误（425 Failed to establish connection）
- Java每日一面(Part2数据库)[19/11/28]
- MySQL日志简介
- 追踪SQL Server执行delete操作时候不同锁申请与释放的过程
- Django实现标签联动以及xadmin中实现标签联动
- Linux第二章-Linux常用命令
- Appium(二)：Node.js下载与安装、非GUI版本appium下载与安装、GUI版本appium下载与安装
- 使用webstrom开发小程序要做的设置
- Java连载55-接口的作用、接口举例