co 1.1 released

English

github

Changes

Code structure adjustment

• Put the header file in the include directory.
• The source files are placed in the src directory.
• rpcgen was renamed to gen.
• Remove the unitest/base directory and put the unit test code directly under the unitest directory.
• Support subdirectories under test directory.

fast

• The definition of static variables in fast.cc is put into functions, and initialization is safer.
• fast::dtoa deprecated the implementation based on LruMap and replaced with the implementation of Milo Yip.
• Added fast::stream class to provide basic streaming output operations.

fastream

• Inherited from fast::stream class.
• Support move constructor.
• Added empty status.

  fastream fs; // Define an empty fastream object without allocating memory

• Support append itself.

  fastream fs;
  fs << "hello "<< 23;
  fs.append(fs);
  fs << fs; // <==> fs.append(fs)

fastring

• Like fastream, it inherits from fast::stream class, so it also supports streaming output operations.

  fastring s;
  s << "hello "<< 23;

• The memory structure of fastring and fastream is the same, the two can be converted seamlessly.

  fastring s;
  fastream& fs = *(fastream*)&s

• Removed reference counting to make fastring's copying behavior similar to std::string, which is not easy to make mistakes.

  fastring s("hello");
  fastring t(s); // Create a new string through memory copy

• append operation adds inside check to fix logic bug on memory overlaps.

  fastring s("123");
  s.append(s.c_str() + 1); // s -> "12323"

• Remove the clone(), it is no more needed as the reference count was removed.

str

• str::dbg() supports std::unordered_map and std::unordered_set.

flag

• Optimize the parsing order of command line parameters and configuration files, first parse the configuration file, and then parse other command line parameters.

  # First parse the configuration file xx.conf, then parse other command line parameters
  # The values ​​of x and s in the command line will override the values ​​in xx.conf for easy debugging
  ./xx -x -s="hello" -config=xx.conf

• Added built-in bool flag daemon on Linux platform to support background running programs

  # add -daemon in the command line parameters
  # or set in the configuration file: daemon = true
  ./xx -daemon

• Command line parameters support multiple styles, -x=y can be written as -x y or x=y

  ./xx -i=8 u=88 -s="hello world"
  ./xx -i 8 -u 88 -s "hello world"

• Optimize the way to specify the configuration file when the program starts.

  ./xx config=xx.conf  # Use flag config to display the specified
  ./xx xx.conf         # The configuration file name ends with .conf or config 
                       # and is the first non-flag parameter, then config= can be omitted
  ./xx -x xx.conf      # -x is the flag, xx.conf is the first non-flag parameter

• When defining the flag, you can specify the level in the comment to control the order of the flag in the configuration file.

  // Use #3 at the beginning of the comment to specify level 3
  // The supported level is 0-99, the default is 10
  // When using --mkconf to automatically generate a configuration file, the flags are sorted by level, file name, and number of lines of code
  DEF_bool(b, false, "#3 xxx");

log

• Some functions in the signal handler are changed to async-signal-safe version functions, which is safer.

coroutine

• Fixed bugs caused by io events registered in epoll(kevent, iocp) that were not removed in time.
• Fix the bug that the internal iterator is not updated correctly when Scheduler::add_timer() is called.
• Improve the implementation of co::connect, co::accept, etc. to support ipv6.
• Added co::max_sched_num() interface to get the maximum number of scheduling threads supported, which is currently the number of CPU cores in the system.
• Added co::sched_id() interface to get current scheduling thread id.
• Added coroutine_id() interface to get the id of current coroutine.
• Refactored Scheduler, the internal logic structure is clearer, and the code is more readable.
• Modify the parameters of the co::tcp_socket(), co::udp_socket() interface to address family, deprecating the earlier way of specifying ipv4 and ipv6 with 4 and 6.

  sock_t tcp_socket(int af=AF_INET); // @af: address family, AF_INET, AF_INET6, etc.
  sock_t udp_socket(int af=AF_INET); // @af: address family, AF_INET, AF_INET6, etc.

• Added co::socket() interface, which is consistent with the native API.
• Fixed the initialization problem of some static global variables in the hook implementation.
• Optimized the internal implementation of co::Event.
• Refactored co::Pool:
  • Users can specify callback ccb and dcb, which are used to create and destroy an element respectively.
  • Users can specify the maximum capacity of the pool (only valid when dcb is set).
  • Register cleanup callback with Scheduler in the internal implementation to ensure the cleanup of co::Pool at the end of the scheduling thread.
• co::Kakalot was renamed to co::PoolGuard.

json

• Internal reference counting, using atomic operations, copying Json objects is safe in multi-thread environment.
• Reconstruct the internal memory model of Json, and fix the bugs caused by the internal memory changes.
• A simple memory allocator Jalloc is added to improve the performance of Json.
• json::parse() supports parsing of array objects.

  Json v = json::parse("[1, 2, 3]");

• Added Json::dbg() interface to convert Json object to debug string (longer strings in Json objects may be truncated).
• The log library calls Json::dbg() to output Json objects, making the output log more streamlined.

rpc

• Simplify, remove some unnecessary configuration items.
• Optimize connection management, you can specify the timeout period of idle connections and the maximum number of idle connections through rpc_conn_idle_sec and rpc_max_idle_conn.

hash

• Modify the implementation of hash32(), the 32-bit system uses the 32-bit version of murmur 2, the 64-bit system directly takes the lower 32 bits of hash64.

Compile

• Removed scons compilation script.
• Support xmake compilation.
• Support cmake compilation (contributed by izhengfan).
• Windows supports compilation with VS project files (automatically generated by xmake).

中文

github

Changes

代码结构调整

• 头文件放到 include 目录.
• 源文件放到 src 目录.
• rpcgen 更名为 gen.
• 移除 unitest/base 目录，单元测试代码直接放到 unitest 目录下.
• test 目录下支持子目录.

fast

• fast.cc 中静态变量的定义放到函数中，初始化更安全.
• fast::dtoa 弃用基于 LruMap 的实现，换用 Milo Yip 的实现(miloyip/dtoa-benchmark).
• 新增 fast::stream 类，提供基本的流式输出操作.

fastream

• 继承于 fast::stream 类.
• 支持 move 构造函数.
• 增加空状态.

  fastream fs; // 定义一个空的 fastream 对象，不分配内存

• 支持 append 自己.

  fastream fs;
  fs << "hello " << 23;
  fs.append(fs);
  fs << fs;     // <==> fs.append(fs)

fastring

• 与 fastream 一样，继承于 fast::stream 类，因此也支持流式输出操作.

  fastring s;
  s << "hello " << 23;

• fastring 与 fastream 的内存结构相同，二者可以无缝转换.

  fastring s;
  fastream& fs = *(fastream*)&s

• 移除引用计数，使 fastring 的复制行为与 std::string 类似，使用起来不容易出错.

  fastring s("hello");
  fastring t(s);  // 通过内存拷贝创建一个新的字符串

• append 操作增加 inside 检查，修复内存重叠时的逻辑漏洞.

  fastring s("123");
  s.append(s.c_str() + 1); // s -> "12323"

• 删除 clone() 方法，移除引用计数后，此方法多余.

str

• str::dbg() 支持 std::unordered_map 与 std::unordered_set.

flag

• 优化命令行参数与配置文件的解析顺序，先解析配置文件，再解析其他命令行参数.

  # 先解析配置文件 xx.conf，再解析其他命令行参数
  # 命令行中 x, s 的值会覆盖 xx.conf 中的值，方便调试
  ./xx -x -s="hello" -config=xx.conf

• Linux 平台增加内置 bool flag daemon，以支持后台运行程序

  # 可在命令行参数中带上 -daemon
  # 也可在配置文件中设置: daemon = true
  ./xx -daemon

• 命令行参数支持多种格式，-x=y 可以写成 -x y 或者 x=y

  ./xx -i=8 u=88 -s="hello world"
  ./xx -i 8 -u 88 -s "hello world"

• 优化程序启动时指定配置文件的方式.

  ./xx config=xx.conf  # 用 flag config 显示指定
  ./xx xx.conf         # 配置文件名以 .conf 或 config 结尾，且是程序命令行的第一个非 flag 参数，则可省略 config=
  ./xx -x xx.conf      # -x 是 flag，xx.conf 是第一个非 flag 参数

• 定义 flag 时，可以在注释中指定级别，以控制 flag 在配置文件中的顺序.

  // 在注释开头用 #3 指定级别为 3
  // 支持的级别为 0-99，默认为 10
  // 使用 --mkconf 自动生成配置文件时，flag 按级别、文件名、代码行数排序
  DEF_bool(b, false, "#3 xxx");

log

• signal handler 中部分函数修改为 async-signal-safe 版本的函数，更安全.

协程库

• 修复未及时移除 epoll(kevent, iocp) 中注册的 io 事件引起的 bug.
• 修复 Scheduler 在 add_timer() 时，内部 iterator 未正确更新的 bug.
• 改进 co::connect, co::accept 等的实现，以支持 ipv6.
• 新增 co::max_sched_num() 接口，获取支持的最大调度线程数，目前为系统 cpu 核数.
• 新增 co::sched_id() 接口，获取当前的调度线程 id.
• 新增 coroutine_id() 接口，获取当前协程的 id.
• 重构 Scheduler，内部逻辑结构更清晰，同时提高代码的可读性.
• 修改 co::tcp_socket(), co::udp_socket() 接口的参数为 address family，弃用早期用 4 与 6 指定 ipv4 与 ipv6 的方式.

  sock_t tcp_socket(int af=AF_INET); // @af: address family, AF_INET, AF_INET6, etc.
  sock_t udp_socket(int af=AF_INET); // @af: address family, AF_INET, AF_INET6, etc.

• 新增 co::socket() 接口，与原生 api 保持一致.
• 修复 hook 实现中一些静态全局变量的初始化问题.
• 优化 co::Event 的内部实现.
• 重构 co::Pool:
  • 用户可以指定 callback ccb 与 dcb，分别用于创建、销毁元素.
  • 用户可以指定 pool 的最大容量(仅在 dcb 设置时有效).
  • 内部实现中向 Scheduler 注册 cleanup callback，保证在调度线程结束时进行 co::Pool 的清理工作.
• co::Kakalot 重命名为 co::PoolGuard.

json

• 内部引用计数，使用原子操作，复制 Json 对象更安全.
• 重构 Json 内部的内存模型，修复之前因内部内存变化引起的 bug.
• 内部增加简单的内存分配器 Jalloc，提升 Json 的性能.
• json::parse() 支持数组对象的解析.

  Json v = json::parse("[1, 2, 3]");

• 新增 Json::dbg() 接口，将 Json 对象转换为 debug string （Json 对象中较长的字符串可能被截断）.
• log 库调用 Json::dbg() 输出 Json 对象，使得输出的日志更精简.

rpc

• 简化，移除一些不必要的配置项.
• 优化连接管理，可以通过 rpc_conn_idle_sec 与 rpc_max_idle_conn 指定空闲连接的超时时间、最大的空闲连接数.

hash

• 修改 hash32() 的实现，32 位系统使用 murmur 2 的 32 位版本，64 位系统直接取 hash64 的低 32 位.

编译

• 移除 scons 编译脚本.
• 支持 xmake 编译.
• 支持 cmake 编译 (由 izhengfan 贡献).
• windows 支持用 vs 工程文件编译 (由 xmake 自动生成).