Pages:  1 2 3 4 5 6 7 ... 21 22
lytsing – Page 3 – lytsing's Blog

Author: lytsing

  • 当心avahi-daemon

    我们的系统时不时报

    java.net.ConnectException: Connection timed out
            at java.net.PlainSocketImpl.socketConnect(Native Method)
            at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:351)
            at java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:213)
            at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:200)
            at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
            at java.net.Socket.connect(Socket.java:529)
            at java.net.Socket.connect(Socket.java:478)
            at java.net.Socket.<init>(Socket.java:375)
            at java.net.Socket.<init>(Socket.java:189)

    这样的错误日志。被连接的server 系统负载不是很高,资源也足够,一时找不到原因。分析日志出现的时间:

    $ grep “router socket IO Exception” router.log |awk ‘{print $2}’|sort -u
    10:15:05,155
    10:15:13,467
    12:15:06,129
    12:15:13,419
    13:15:06,012
    13:15:06,148
    13:15:07,168
    13:15:08,812
    13:15:11,781
    13:15:14,154
    13:15:14,510
    15:15:05,306
    15:15:05,639
    16:15:06,709
    16:15:07,083
    16:15:07,447
    16:15:07,850
    16:15:07,915
    16:15:08,888
    16:15:10,359
    16:15:12,553
    16:15:14,246
    17:15:08,219
    17:15:08,239
    17:15:09,055
    17:15:12,094
    18:15:06,008
    18:15:08,460
    18:15:10,428
    18:15:12,057
    18:15:12,692
    19:15:11,374
    19:15:12,061
    19:15:12,089
    19:15:13,746
    21:15:08,496
    21:15:11,356
    21:15:12,393
    21:15:13,462
    21:15:14,495

    都出现在神秘的15分钟,应该是定时jobs在跑导致的。检查crontab以及jboss的jobs任务,也找不到。最后只能tcpdump在14-16分抓包分析。得知 server 启用了avahi-daemon服务,每隔一个小时发送一个MDNS的广播包,会造成 server 无回应20秒左右。

    avahi-daemon 是什么? 来自 http://packages.debian.org/zh-cn/wheezy/avahi-daemon 的解释: Avahi 是一个完全基于 LGPL 框架下的多播 DNS 服务搜寻工具。它能让程序发布或者 发现在本地网络中的服务或者主机,而无需特殊配置。例如,您可以连入一个网络,然后 立刻发现可用于打印的打印机,可查看的文件或者可聊天的朋友。本软件包包含 Avahi 守护进程,它将在网络中展示您的机器,并允许其他应用程序发 布和分析 mDNS/DNS-SD 记录.

    服务器上最好把 avahi-daemon 关闭掉。

  • 吐槽一下新浪微博 Android SDK V2.4.0

    吐槽一下新浪微博 Android SDK V2.4.0

    周末逛逛github,发现新浪更新了SDK,说是重大版本变更,瞄了一下,大概是文档更规范、代码重构之类的。老版本SDK代码全开放,但 2.3之后,封闭了部分代码。不管如何,我还是直奔观看 AsyncWeiboRunner.request 这个方法究竟改变了没有,结果还是失望了。

    我建议微博 Android SDK 的开发人员,好好的阅读AQuery,Volley,android-async-http 这几个异步网络请求库,通过回调方式处理请求结果。新浪也是有回调结果,但要自己加 Handler 更新UI,官方的Demo在回调直接更新UI,其实就是在 http 请求线程里操作的,很容易给新手误导。

    很多软件的开发都是如此,开始使用现成的,后面发现使用越来越不爽,不能满足自己的需求,决定重新搞。所以,Weibo for Android 计划使用 Volley作为http 处理库,自己写封装接口。我封装了一个GsonRequest,测试发现老是返回403,抓包才知道参数没有上传。使用 http get 传参数需要注意,重写getParams无效。见:

    http://stackoverflow.com/questions/18484647/volley-does-not-call-getparams-for-my-custom-request

    getParams() is not called on the GET method, so it seems you’ll have to add it to the URL before you send the request

  • 360 wifi 2 做无线网卡(Linux)

    360 wifi刚出来的时候,抢购几次没拿到,后面需求也不是很大,公司、家里都有无线路由器。现在公司人多了,公用的wifi网络变得超级慢,而且mac pro的网线口好像坏了,无法直接插线。琢磨了一会,在我的PC机上装个 360 wifi做 AP。开始以为可以在Linux 虚拟机的xp系统里使用,但当装驱动时,系统崩溃,放弃。Google了 360 wifi linux,发现只有两篇个链接可参考:

    360 wifi 2 硬件信息情况:
    [deli@violet ~]$ lsusb
    Bus 002 Device 009: ID 148f:7601 Ralink Technology, Corp.

    $ lspci
    04:00.0 Ethernet controller: Realtek Semiconductor Co., Ltd. RTL8111/8168B PCI Express Gigabit Ethernet controller (rev 06)

    得装驱动,从MTK网站下载

    tar xvf DPO_MT7601U_LinuxSTA_3.0.0.4_20130913.tar.bz2
    cd DPO_MT7601U_LinuxSTA_3.0.0.4_20130913/
    sudo make install
    sudo modprobe mt7601Usta

    su -
    echo 148F 7601 > /sys/bus/usb/drivers/rt2870/new_id
    exit

    可以查看多了一个网卡口 ra0:

    [deli@violet ~]$ ifconfig
    lo Link encap:Local Loopback
    inet addr:127.0.0.1 Mask:255.0.0.0
    inet6 addr: ::1/128 Scope:Host
    UP LOOPBACK RUNNING MTU:16436 Metric:1
    RX packets:292 errors:0 dropped:0 overruns:0 frame:0
    TX packets:292 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:0
    RX bytes:22920 (22.3 KiB) TX bytes:22920 (22.3 KiB)

    p2p1 Link encap:Ethernet HWaddr F4:6D:04:73:D4:A6
    inet6 addr: fe80::f66d:4ff:fe73:d4a6/64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:66065 errors:0 dropped:0 overruns:0 frame:0
    TX packets:44804 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:38222311 (36.4 MiB) TX bytes:5841642 (5.5 MiB)

    ra0 Link encap:Ethernet HWaddr 00:87:36:12:0E:81
    inet addr:172.30.3.30 Bcast:172.30.3.255 Mask:255.255.255.0
    inet6 addr: fe80::287:36ff:fe12:e81/64 Scope:Link
    UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
    RX packets:144725 errors:0 dropped:0 overruns:0 frame:0
    TX packets:1177 errors:0 dropped:0 overruns:0 carrier:0
    collisions:0 txqueuelen:1000
    RX bytes:41587475 (39.6 MiB) TX bytes:127539 (124.5 KiB)

    $ ethtool -i ra0
    driver: RALINK WLAN
    version:
    firmware-version:
    bus-info: CSR 0x0
    supports-statistics: no
    supports-test: no
    supports-eeprom-access: no
    supports-register-dump: no

    MT7601的Linux驱动中没有实现nl80211的接口,hostapd没法直接用。下面是尝试的过程:
    我的Linux是 fedora 16,360 wifi 2, 需要对 github上的脚本做修改:
    https://gist.github.com/lytsing/991ab69460587aa91aa5
    $ ./ubuntu.sh ra0 p2p1 password

    [deli@violet 360-wifi-linux]$ sudo hostapd ~/.360wifi/.hostapd.ra0.conf
    Configuration file: /home/deli/.360wifi/.hostapd.ra0.conf
    nl80211: 'nl80211' generic netlink not found
    nl80211 driver initialization failed.

    加载 mac80211 再试试。
    [deli@violet 360-wifi-linux]$ sudo modprobe mac80211

    [deli@violet 360-wifi-linux]$ sudo hostapd -dd ~/.360wifi/.hostapd.ra0.conf

    Configuration file: /home/deli/.360wifi/.hostapd.ra0.conf
    nl80211: Failed to set interface ra0 into AP mode
    nl80211 driver initialization failed.
    ELOOP: remaining socket: sock=4 eloop_data=0x9b77898 user_data=0x9b77da8 handler=0x8075380
    ELOOP: remaining socket: sock=6 eloop_data=0x9b795c0 user_data=(nil) handler=0x807ef50

    无解。

  • 写脚本群发薪水清单邮件通知

    一到发薪水日,我们的人事忙得不可开交,给每位员工发送上月的薪水清单邮件,一个个从Excel表格复制粘贴到邮件,忙到晚上10点才结束,整得人精疲力尽。HR让我帮她解决这个难题,她说她用foxmail,网上搜索,应该有一些工具可以完成。但我对foxmail,outlook又不是很熟悉,想想还是使用自己擅长的脚本写。脚本这东西,长时间不练习,就生疏了,整整花了早上2小时,刚好200行代码:)为了学习python,这次硬着头皮用python写,在 windows xp, windows 8 , mac OS X 10.8.4 测试通过。

    send_salary_mail.py:

    # -*- coding: utf-8 -*-
    #!/usr/bin/env python2.7
    
    '''
     Copyright 2013 IBOXPAY Inc
     Description: this script help HR send mail to all staffs.
     Dependency: xlrd, Install:
     wget --no-check-certificate https://pypi.python.org/packages/source/x/xlrd/xlrd-0.9.2.tar.gz
     tar xvf xlrd-0.9.2.tar.gz
     cd xlrd-0.9.2
     sudo python setup.py install
    
     Written by Lytsing Huang 2013-09-12
     refer: http://code.activestate.com/recipes/578150-sending-non-ascii-emails-from-python-3/
    '''
    
    import sys
    import smtplib
    from email.header import Header
    from email.mime.text import MIMEText
    from email.utils import formataddr
    from datetime import datetime
    import xlrd
    
    reload(sys)
    sys.setdefaultencoding('utf-8')
    
    mail_host = 'mail.xxx.com' #发送邮件的smtp地址
    mail_user = 'xxx@xxx.com' # 发送通知邮件的用户名
    mail_pass = 'xxx' # 用户的密码
    
    sender_name = 'xxx'
    sender_addr = 'xxx@xxx.com'
    subject = '2013年8月XXX公司薪水发放通知单' # ***邮件标题*** 每月需要手动修改年月
    title = subject
    
    html_template = """
    <html>
    <head>
    <meta http-equiv="content-type" content="text/html;charset=utf-8" />
    </head>
    <body>
    <h2 align="center">%s</h2>
    <table border="1", cellpadding="2" cellspacing="0">
        <thead>
            <tr>
                <th>姓名</th>
                <th>基本工资</th>
                <th>岗位津贴</th>
                <th>交通补贴</th>
                <th>补助天数</th>
                <th>补助</th>
                <th>绩效工资</th>
                <th>其他</th>
                <th>应发合计</th>
                <th>公司承担社保
                <th>公司承担住房公积金</th>
                <th>代扣社保</th>
                <th>代扣住房公积金</th>
                <th>病假天数</th>
                <th>病假</th>
                <th>事假天数</th>
                <th>事假</th>
                <th>个税</th>
                <th>其他扣款</th>
                <th>实发工资<th>
            </tr>
        </thead>
        <tbody>
            <tr>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
                <td> %s </td>
            </tr>
        </tbody>
    </table>
    
    <pre>
     说明:
     1、以上薪水将会转入您提供给公司的银行账户中,如银行账户有更改请及时通知人力资源部。
     2、员工薪水信息属公司机密,请妥善保管,泄漏者将按公司有关制度处理。
     3、如对薪水支付数额有异议的,请于一周内以邮件的形式向人力资源部提出。
    
     人力资源部
     %s
    
      ----------------------------------------------
    
                  %s    人事
    </pre>
    </body>
    </html>
    """
    
    def write_mail_content(header_title, name, base_salary, post_allowance,
            transport_subsidies, grants_days, allowance, pay_for_performance,
                        other, total, company_pension_commitment, company_housing_fund_commitment,
                        withholding_social_security, withholding_housing_fund, sick_days,
                        sick_leave, leave_days, leave, personal_income_tax, other_withholding,
                        real_wages, today, sender):
    
        content = html_template % (header_title, name, base_salary, post_allowance,
                transport_subsidies, grants_days, allowance, pay_for_performance,
                other, total, company_pension_commitment, company_housing_fund_commitment,
                withholding_social_security, withholding_housing_fund, sick_days,
                sick_leave, leave_days, leave, personal_income_tax, other_withholding,
                real_wages, today, sender)
    
        return content
    
    def send_mail(msg, sender, recipient):
        try:
            s = smtplib.SMTP()
            s.connect(mail_host)
            s.ehlo()
            s.starttls()
            s.login(mail_user,mail_pass)
            s.sendmail(sender, recipient, msg.as_string())
            s.close()
            return True
        except Exception, e: print str(e)
        return False
    
    def write_mail(sender, recipient, sub, content):
        name = Header(sender, 'utf-8').encode()
        msg = MIMEText(content, _subtype = 'html', _charset='utf-8')
        msg['Subject'] = Header(sub, 'utf-8')
        msg['From'] = formataddr((name, sender_addr))
        msg['To'] = recipient
        return msg
    
    def main():
    
        if  len(sys.argv) < 2:
            print '错误,没有指定参数'
            print '用法:python send_salary_mail.py xxx.xls'
            sys.exit()
    
        bk = xlrd.open_workbook(sys.argv[1])
        #bk.sheets()返回一个列表
        sh = bk.sheets()[0]  #读取第一张sheet
        #下面是按行读取excel表格内容
        for row in range(3, sh.nrows):
            name = sh.row(row)[2].value # 姓名
            recipient_addr = sh.row(row)[3].value # Email
            base_salary = sh.row(row)[7].value # 基本工资
            post_allowance = sh.row(row)[8].value # 岗位津贴
            transport_subsidies = sh.row(row)[9].value # 交通补贴
            grants_days = sh.row(row)[10].value # 补助天数
            allowance = sh.row(row)[11].value # 补助
            pay_for_performance = sh.row(row)[12].value # 绩效工资
            other = sh.row(row)[13].value # 其他
            total = sh.row(row)[14].value # 应发合计
            company_pension_commitment = sh.row(row)[15].value # 公司承担社保
            company_housing_fund_commitment = sh.row(row)[16].value # 公司承担住房公积金
            withholding_social_security = sh.row(row)[17].value # 代扣社保
            withholding_housing_fund = sh.row(row)[18].value # 代扣住房公积金
            sick_days = sh.row(row)[19].value # 病假天数
            sick_leave = sh.row(row)[20].value # 病假
            leave_days = sh.row(row)[21].value # 事假天数
            leave = sh.row(row)[22].value # 事假
            personal_income_tax = sh.row(row)[24].value # 个税
            other_withholding = sh.row(row)[25].value # 其他扣款
            real_wages = sh.row(row)[26].value # 实发工资
            today = datetime.now().strftime('%Y/%m/%d')
    
            content = write_mail_content(title, name, base_salary, post_allowance,
                    transport_subsidies, grants_days, allowance, pay_for_performance,
                    other, total, company_pension_commitment, company_housing_fund_commitment,
                    withholding_social_security, withholding_housing_fund, sick_days,
                    sick_leave, leave_days, leave, personal_income_tax, other_withholding,
                    real_wages, today, sender_name)
    
            msg = write_mail(sender_name, recipient_addr, subject, content)
            if send_mail(msg, sender_addr, recipient_addr):
                print ' 姓名:' + name + ' 发送成功.'
            else:
                print ' 姓名:' + name + ' 发送失败.'
    
        print 'Send all finished! please check out the failed records.'
    
    if __name__ == '__main__':
        main()

    工资表测试样例:点这下载

    有需要的朋友可以参考下。

  • Android、iOS http请求加上User-agent

    最近在排查一些问题,发现很难下手,nginx 的log如下:

    182.37.109.153 - - [25/Aug/2013:00:02:52 +0800] "-" 400 0 "-" "-" "-"
    182.37.109.153 - - [25/Aug/2013:00:02:52 +0800] "-" 400 0 "-" "-" "-"
    182.37.109.153 - - [25/Aug/2013:00:02:52 +0800] "-" 400 0 "-" "-" "-"
    171.36.8.66 - - [25/Aug/2013:09:40:07 +0800] "POST /xxxxx.htm HTTP/1.1" 200 251 "-" "-" "-"

    不知道哪个请求是Android,哪个是iOS的。Square的Jack在演讲中提到过,他们最开始做后台系统,基于twitter有过的经验,第一个系统不是实现后台交易功能,而是控制台,先做好监控用户的每个交易行为,使得可控。

    iOS 如果使用 ASIHTTPRequest 库,ASIHTTPRequest.m 里的代码如下:

    [self setDefaultUserAgentString:[NSString stringWithFormat:@”%@ %@ (%@; %@ %@; %@)”, appName, appVersion, deviceName, OSName, OSVersion, locale]];
    得到默认的 User-agent: “miniCashbox 2.0.6 (iPhone; iPhone OS 6.1.3; zh_CN)”

    Android可以使用系统提供的,需要设置
    HttpProtocolParams.setUserAgent(params, System.getProperty(“http.agent”));
    结果为:User-agent: “Dalvik/1.6.0 (Linux; U; Android 4.1.2; Nexus S Build/JZO54K)”

    根据实际需要,以上的 User-agent 还不足够满足需要,比如想要知道是哪个用户发送的请求,但不可能把用户id显示出来,那么就需要显示设备的唯一标识 Device Id。系统设计时,用户与终端设备信息的关系是: user has_many devices。从常用的QQ、微信在移动终端登录体验过程中,我们可以感受到的。给客户端做消息push推送,更需要如此做。

    参考: http://cdrussell.blogspot.com/2012/09/programmatically-get-user-agent-string.html