博客

  • 关于C/C++内存管理一些乱讲

    http://www.cnblogs.com/skynet/archive/2010/12/03/1895045.html 这篇博客最后写了5个规则,虽然简单,但是还是有些问题,在这里稍作说明。

    【规则1用malloc或new申请内存之后,应该立即检查指针值是否为NULL。防止使用指针值为NULL的内存。

    偶评:参考wiki百科针对malloc以及new的说明,对于C语言malloc方式,检查NULL是可以的,但是对于C++的new操作符(operator),检查NULL基本是无用的,因为C++有异常机制,new不成功就会抛异常std::bad_alloc,如何处理可参考 http://msdn.microsoft.com/en-us/library/kftdy56f%28v=VS.71%29.aspx

    【规则2不要忘记为数组和动态内存赋初值。防止将未被初始化的内存作为右值使用。

    偶评:初值的赋值操作也是要花时间的,C语言初始化一块内存memset的时候要注意用buffersize * sizeof(Object),而C++就更复杂了,比如构造函数,new的重载之类,要小心。

    【规则3避免数组或指针的下标越界,特别要当心发生“多1”或者“少1”操作。

    偶评:一般来讲多一少一运行时不会一定出错,比如微软编译器申请内存一般会比你指定的大一些,比如你申请10bytes,如char* p= malloc(10);,通常情况下操作p[10],p[11]啥的是没有问题的,所以会发生一些奇怪的现象,就是某些bug有人机器会出,有人就无法重现。

    【规则4动态内存的申请与释放必须配对,防止内存泄漏。

    偶评:要说明的是,new一定和delete配对,malloc一定和free配对,否则也会出错。另外new [],一定要delete [],否则也会泄露。

    【规则5用free或delete释放了内存之后,立即将指针设置为NULL,防止产生“野指针”。

    偶评:这种情况一般是代码中函数写的比较长,指针用了又用,或者指针是全局的情况。所以一般软件公司的编码规范都会硬性要求这样做,如果函数很短小精悍,其实不会有这种担忧的。当然,偶也同意这样做有益无害。

    再加入一些额外的说明:

    对于指针类型的检查,不应该用assert,而是应该正常操作,if(!p) return E_POINTER;这样。使用assert的语义与空指针判断是两码事。如果不想判断或者图省事,C++中可以用const reference类型,但是不建议传入reference对象,然后在函数体内修改。

    针对内存操作,比如memset,memcpy,同样要注意不能访问越界的数据,类似规则3。

    针对C++,new delete其实可以做出好多花样,个人感觉用处不大,因为服务器端编程大多用C配合内存池,重载new之类的学了也很少用。如果对服务器端内存管理感兴趣,可以读读nginx的代码,简洁高效实用。

    一般来讲,应该是谁拉的屎谁擦屁股,也就是说某个对象或者函数new了一些东西,它应该负责delete,否则距离不仅仅产生美,还会产生bug。(多废话一下,比如在aaa.cpp里面new了一些东西,然后你在bbb.cpp里面释放它,甚至在不同地方释放它,如果对程序结构不了解,很容易产生bug)

    可以通过python或者lua编写一些小软件,检查项目中new与delete,malloc与free的个数,如果个数不匹配,就有可能有bug,另外还可以检查new[]与delete[]的配对情况。我编写了一个这样的小工具,完善以后放出来。

  • A Fool with a Tool is still a Fool

    这句话真的是太经典了,让我有冲动写篇博客赞颂一下。

    http://www.dwheeler.com/flawfinder/

    flawfinder是一个c/c++代码静态分析工具,类似splint,或者pclint,尽管在VisualC++找不到这个工具,其实VC++已经自带了,在企业版我们可以定义代码分析扫描规则,或者一般的版本,但你把Build Warning调到Level4,也可以看到足够的信息(如果做C++编程,没有硬性要求把Warning级别调到最高,相信我,产品质量非常值得怀疑),据我所知GCC也可以调整Warning级别,一般来说,消除了Warning提示的,代码质量可以达到50分,满分100。

    我们抛开对FlawFinder这个工具的评价,只看关于这句话的解释。

    Any static analysis tool, such as Flawfinder, is merely a tool. No tool can substitute for human thought! In short, "a fool with a tool is still a fool". It’s a mistake to think that analysis tools (like flawfinder) are a substitute for security training and knowledge. Developers – please read documents like my Secure Programming book so you’ll understand the vulnerabilities that the tool is trying to find! Organizations – please make sure your developers understand how to develop secure software (including learning about the common mistakes past developers have made), before having them develop software or use static analysis tools.

    An example of horrific tool misuse is disabling vulnerability reports without (1) fixing the vulnerability, or (2) ensuring that it is not a vulnerability. It’s publicly known that RealNetworks did this with flawfinder; I suspect others have misused tools this way. I don’t mean to beat on RealNetworks particularly, but it’s important to apply lessons learned from others, and unlike many projects, the details of their vulnerable source code are publicly available. As noted in iDEFENSE Security Advisory 03.01.05 on RealNetworks RealPlayer (CVE-2005-0455), a security vulnerability was in this pair of lines:

     char tmp[256]; /* Flawfinder: ignore */ strcpy(tmp, pScreenSize); /* Flawfinder: ignore */

    This means that flawfinder did find this vulnerability, but instead fixing it, someone added the "ignore" directive to the code so that flawfinder would stop reporting the vulnerability. But an "ignore" directive simply stops flawfinder from reporting the vulnerability – it doesn’t fix the vulnerability! The intended use of this directive is to add it once a reviewer determined that it was definitely a false positive, but in this case the tool was reporting a real vulnerability. The same thing happened again in iDefense Security Advisory 06.23.05, where the vulnerable line was:

     sprintf(pTmp, /* Flawfinder: ignore */

    And a third vulnerability with the same issue was reported still later in iDefense Security Advisory 06.26.07, RealNetworks RealPlayer/HelixPlayer SMIL wallclock Stack Overflow Vulnerability, where the vulnerable line was:

     strncpy(buf, pos, len); /* Flawfinder: ignore */

    This is not to say that RealNetworks is a fool or set of fools. Indeed, I believe many organizations, not just RealNetworks, have misused tools this way. My thanks to RealNetworks publicly admitting their mistake – it allows others to learn from their mistake! My specific point is that you can’t just add comments with "ignore" directives and expect that the software is suddenly more secure. Do not add "ignore" directives until you are certain that the report is a false positive.

    This kind of problem can easily happen in organizations that say "run scanning tools until there are no more warnings" but don’t later review the changes that were made to eliminate the warnings. If warnings are eliminated because code is changed to eliminate vulnerabilities, that’s great! General-purpose tools scanning like flawfinder will have false positive reports, though; it’s easy to create a tool without false positives, but they’ll do that by failing to report many possible vulnerabilities (some of which will really be vulnerabilities). The obvious answer if you want a broader tool is to allow developers to examine the code, and if they can truly justify that it’s a false positive, document why it is a false positive (say in a comment near the report) and then add a "Flawfinder: ignore" directive. But you need to really justify that the report is a false positive; just adding an "ignore" directive doesn’t fix anything! Sometimes it’s easier to fix a problem that may or may not be a vulnerability, instead of ensuring that it’s a false positive – the OpenBSD developers have been doing this successfully for years, since if complicated code isn’t an exploitable vulnerability yet, a tiny change can often turn such fragile code into a vulnerability.

    If you’re in an organization using a scanning tool like this, make sure you review every change caused by a vulnerability report. Every change should be either (1) truly fixed or (2) correctly and completely justified as a false positive. I think organizations should require any such justification to be in comments next to the "ignore" directive. If the justification isn’t complete, don’t mark it with an "ignore" directive. And before developers even start writing code, get them trained on how to write secure code and what the common mistakes are; this material is not typically covered in university classes or even on the job.

    原文并不复杂,我在这就不翻译了。我同意作者提出的这个观点,对于傻瓜来说,再好的工具也白扯。

    最近大力鼓吹推行Code Review、Unit Test,其实经常看我博客的朋友都知道,我一直在讲这句话,提升代码质量的两个关键方法就是Code Review加上单元测试,也在自己的项目中身体力行,不过由于我只是小兵一个,所以只能自己做,还打不到让其他人也跟着做的程度。对于这两个工具,没有好的执行力推动,其实是不可能达到提升软件质量的效果的。用工具,用技术,但是最关键的一点是要用对工具,而且要有持之以恒的执行。

    最简单的也最容易的就是从手头的项目开始,如果编译工具可以调整Warning,调到最高,然后fix掉这些Warning。如果遇到Bug,不要Ignore,先总结Root Cause,然后写Test case,改代码,编译,跑单元测试。只要持之以恒,不需要什么CEO总经理推动,产品质量自然就提升上来了。

    如果你不是傻瓜,但是你的组员是这样的傻瓜该怎么办?如果你有权力,建议你把他调开,有些人真的是不适合编程,不必强求,也许他搞销售更适合呢,让这种傻瓜捣乱的结果就是让整个开发组的士气低落;如果你没法让他离开,那就让他做一些边缘性的项目,不要碰关键代码,另外要小心盯着傻瓜添加或者修改的代码,因为一颗老鼠屎会坏了一锅汤,而且很有可能这个黑锅会被你背上。

  • 一些技术摘选及随想

    code review的目标,1)消除所有W4警告 2)给不能直接理解的代码加注释 3)丑陋代码的修改,主要针对过多for/while/if/switch这类控制的互相嵌套,把它们削平

    https://bugzilla.redhat.com/show_bug.cgi?id=638477 Linus随手写了个memcpy,当然你可以在glib或者dietlibc这些标准库实现中找到工程级强度代码。

    Python web server方案选择比较http://imilky.cn/blog/2010/04/python-wsgi-web-server/ 另外可以考虑http://mongrel2.org/ 现在这个新版本支持Python、Lua、Ruby、C++,uwsgi可以使用Lua开发 http://projects.unbit.it/uwsgi/wiki/Lua

    如何测试webserver性能,这篇可以作为开始阅读的起点:http://en.wikipedia.org/wiki/Web_server_benchmarking 

    另外很多测试提到ab,其实就是apachebench。

    如何让node.js与apache/nginx共用,如何让tornadoweb与apache/nginx共用,用不着那些有的没的复杂步骤,apache用rewrite,nginx考虑proxy_pass就都完事了。

    C语言正则表达式库,可以用libc里面的,也可以用pcre,oniguruma。

    fastcgi可以用C写,自己devkit就有范例 http://www.fastcgi.com/devkit/examples/authorizer.c

    http://hyperpolyglot.org/lisp Lisp: Common Lisp, Scheme, Clojure, Emacs Lisp lisp各式方言对比!

    http://hyperpolyglot.org/scripting Scripting Languages: PHP, Perl, Python, Ruby, Smalltalk 脚本语言特性比较cheatsheet。

    http://hyperpolyglot.org/small Scripting Languages: Bash, Tcl, Lua, JavaScript, Io 特性对比

    http://hyperpolyglot.org/c C, C++, Objective C, Java, C#特性对比cheatsheet

    C语言装逼教程推荐,http://sunxiunan.com/?p=1661 看后面书籍推荐部分,另外只需要加上一本《高效程序的奥秘》

    建议Python相关讲师多使用appengine,uliweb多加入一些相关示例,appengine又易用又免费又容易部署,那些迁移成本、不能完成的任务先不考虑,用appengine做一些力所能及的东西,让其他人能实际用到,这种成就感是对初学者最好的鼓励。

    最后一条推荐的新闻是Lua5.2进入rc alpha阶段,当然距离真正release也许还要不少时间,但是很值得关注!

  • Lua cheat sheet

    lua_cheat_sheet

    http://coffeeghost.net/images/lua_cheat_sheet.png

    做一些简单说明。

    关于所有数字都是double,在《programming in lua》第二版中有详细说明。除了极少数情况,可以满足大部分数值计算的需求。

    对于table的说明比较详细。但是对于一些高阶问题没有涉及。比如协程,算是Lua的一大特色。还有函数作为first-class value,也可以做出很多有意思的东西。另外还有meta-table,是Lua模拟OO的关键。另外限于篇幅C API也没有介绍,相比其它编程语言,Lua与C的配合几乎可以说是无缝的。

    相比而言,维基百科上对于Lua的介绍更为全面,而且篇幅也不长。感兴趣的可以参考这个链接作为补充 http://en.wikipedia.org/wiki/Lua_%28programming_language%29

  • lua用于web开发的服务器配置

    有两种方式,一种是apache2.3以上会内置的lua module,大家可以下载apache httpd 2.3.8的代码,在modules目录下有lua这个目录。

    另外一种是今天要介绍的,使用wsapi方式。

    我们使用ubuntu服务器,先确保lua5.14以及apache2都安装成功。

    然后

    sudo apt-get install apache2-mpm-worker liblua5.1-0-dev luarocks

    sudo apt-get install libfcgi-dev libapache2-mod-fcgid

    sudo luarocks install wsapi-fcgi

    然后修改.htaccess或者httpd.conf或者你的vhost配置,添加下面部分。

    Options ExecCGI
    AddHandler fcgid-script .lua
    FCGIWrapper /usr/local/lib/luarocks/bin/wsapi.fcgi .lua

    要注意的是wsapi.fcgi也许是在不同目录下,用find自己找吧。

    在var/www下你的站点中新建一个luacgi目录,然后建立两个文件。

    launcher.fcgi:

    #!/usr/bin/env lua

    require "wsapi.fastcgi"
    require "hello"
    wsapi.fastcgi.run(hello.run)

    index.lua:

    module(…, package.seeall)

    function run(wsapi_env)
      local headers = { ["Content-type"] = "text/html" }

      local function hello_text()
        coroutine.yield("<html><body>")
        coroutine.yield("<p>Hello Wsapi!</p>")
        coroutine.yield("<p>PATH_INFO: " .. wsapi_env.PATH_INFO .. "</p>")
        coroutine.yield("<p>SCRIPT_NAME: " .. wsapi_env.SCRIPT_NAME .. "</p>")
        coroutine.yield("</body></html>")
      end

      return 200, headers, coroutine.wrap(hello_text)
    end

    然后用chown –R www-data:www-data luacgi修改目录owner。

    这时候应该就能用xxx.com/luacgi/index.lua访问了。

    如果你用nginx,也有现成的lua mod可以使用(作者是淘宝的程序员),这里就不多说了。