Erlang 集群中Mnesia Table 如何在不影响正常业务的情况下增加/减少字段?


预备知识

情景描述

如何使用mnesia:transform_table/4在Mnesia Table增加(或减少)字段?
假如我们有一个用于监控记录节点各种情况的app,命名这counter, 每秒要证明一下自己还活着,把计数 + 1。

-record(node_info, {name, count}).
{atomic,ok} = mnesia:create_table(node_info,
[{attributes, record_info(node_info)},
{ram_copies, [node()]}]);

使用一个gen_server每秒计数 + 1.

update_count() ->
Fun = fun() ->
[Client] = mnesia:read(?TABLE, node()),
mnesia:write(Client#node_info{count = Client#node_info.count + 1})
end,
{atomic, ok} = mnesia:sync_transaction(Fun).

mnesia node_info table 记录逻辑完整代码可见
这样,我们先release一下0.1.0版本的包,并把它跑起来
我们把release并启动的流程写在脚本./script/0.1.0-alice-start.sh里。

#!/bin/bash
rm -rf _build
git checkout mnesia-upgrade-0.1.0-alice
export RELX_REPLACE_OS_VARS=true
export NODENAME=alice@127.0.0.1
export COOKIE=zhongwen
rebar3 release
_build/default/rel/counter/bin/counter-0.1.0-alice console

我们把release的包叫做counter-0.1.0-alice并使用git在这些代码上打上mnesia-upgrade-0.1.0-alice的Tag(方便后面切换版本)
关于RELX_REPLACE_OS_VARS的说明请看rebar3 dynamic configuration
release完成后以console的方式启动。
启动后可以看到在console里面每5秒打印一次node_info表里面的内容。

=PROGRESS REPORT==== 13-Nov-2016::13:53:36 ===
application: sasl
started_at: 'alice@127.0.0.1'
Eshell V7.3 (abort with ^G)
(alice@127.0.0.1)1> [[{node_info,'alice@127.0.0.1',2}]]
[[{node_info,'alice@127.0.0.1',3}]]
[[{node_info,'alice@127.0.0.1',4}]]
...

需求升级(单节点时增加字段smp_support)

现在需要增加再加一个字段叫smp_support来记录节点是否支持smp

1.增加smp_support字段

-record(node_info, {name, count, smp_support}).

2.增加transform函数来改变表的结构和内容

-module(counter_db_transform).
-export([add_smp_support_to_node_info_table/0]).
-export([del_smp_support_to_node_info_table/0]).
add_smp_support_to_node_info_table() ->
Fun =
fun({node_info, Node, Count}) ->
{node_info, Node, Count, undefined};
(Record) -> Record
end,
NewAttrList = [node, count, smp_support],
{atomic, ok} = mnesia:transform_table(node_info, Fun, NewAttrList),
ok.
del_smp_support_to_node_info_table() ->
Fun =
fun({node_info, Node, Count, _}) ->
{node_info, Node, Count};
(Record) -> Record
end,
NewAttrList = [node, count],
{atomic, ok} = mnesia:transform_table(node_info, Fun, NewAttrList),
ok.
  • add_smp_support_to_node_info_table/0 用于升级
  • del_smp_support_to_node_info_table/0 用于降级

http://erlang.org/doc/man/mnesia.html#transform_table-4
Argument Fun can also be the atom ignore, which indicates that only the metadata about the table is updated. Use of ignore is not recommended, but included as a possibility for the user do to an own transformation.

这里的ignore就是只改变表的结构,但是不会对返回ignore的值作任何修改,也就是还是原来的record,我们并没有用到。

3.升级的appup.src我们定义的规则为

{"0.2.0",
[
{"0.1.0", [
{load_module, counter_db_schema},
{add_module, counter_db_transform},
{apply, {counter_db_transform, add_smp_support_to_node_info_table, []}},
{update, counter_collect_server, {advanced, delete_id}}
]}
],
[
{"0.1.0", [
{apply, {counter_db_transform, del_smp_support_to_node_info_table, []}},
{delete_module, counter_db_transform},
{load_module, counter_db_schema},
{update, counter_collect_server, {advanced, add_id}}
]}
]
}.

具体的appup.src规则(load_module, add_module, delete_module, update, apply)可以参照cookbook
全部的代码都已完成,把代码打上mnesia-upgrade-0.2.0-alice的Tag,
接下来操作一下如何升级counter-0.1.0-alice到counter-0.2.0-alice。
升级脚本写在./script/upgrade-0.1.0-alice-to-0.2.0-alice.sh

#!/bin/bash
git checkout mnesia-upgrade-0.2.0-alice
export RELX_REPLACE_OS_VARS=true
export NODENAME=alice@127.0.0.1
export COOKIE=zhongwen
rebar3 release
rebar3 relup
rebar3 tar
mv _build/default/rel/counter/counter-0.2.0-alice.tar.gz _build/default/rel/counter/releases/0.2.0-alice/counter.tar.gz
_build/default/rel/counter/bin/counter-0.1.0-alice upgrade 0.2.0-alice

相比于0.1.0的release完整包的操作,我们在打0.2.0包是使用的升级包的方式,
release => relup => tar => 使用bin/counter-0.1.0-alice upgrade 0.2.0-alice升级。
执行此脚本后结果如下

===> tarball /Users/zhongwen/github/erlang-rock/mneisa-cluster-hot-upgrade-record-structure/counter/_build/default/rel/counter/counter-0.2.0-alice.tar.gz successfully created!
Release 0.2.0-alice not found, attempting to unpack releases/0.2.0-alice/counter.tar.gz
Unpacked successfully: "0.2.0-alice"
Installed Release: 0.2.0-alice
Made release permanent: "0.2.0-alice"

说明升级成功,我们再来去上面的console里面看一下每5秒钟打印的输出有木有了smp_support字段。

[[{node_info,'alice@127.0.0.1',124}]]
[[{node_info,'alice@127.0.0.1',125}]]
code change {169265188944133434916933618072298650122,
{state,undefined,5000},
delete_id}
[[{node_info,'alice@127.0.0.1',126,true}]]
[[{node_info,'alice@127.0.0.1',127,true}]]
[[{node_info,'alice@127.0.0.1',128,true}]]
(alice@127.0.0.1)6> mnesia:table_info(node_info, attributes).
[node,count,smp_support]
(alice@127.0.0.1)7> application:which_applications().
[{sasl,"SASL CXC 138 11","2.7"},
{counter,"counter","0.2.0"},
{mnesia,"MNESIA CXC 138 12","4.13.3"},
{stdlib,"ERTS CXC 138 10","2.8"},
{kernel,"ERTS CXC 138 10","4.2"}]

升级成功!
PS: code change发生的原因是因为我们使用了appup.src里面使用update规则把state里面的id删掉了,有兴趣可以细看一下0.1.0升级和0.2.0的代码区别(在release如何运用code_change/3 热升级gen_server里面的state)。

需求再升级(在集群节点中再增加字段用于统计每个节点binary使用情况的binary)

counter applications的原来的目的就是集群中每个节点都把自己的情况以node()为key存到node_info表里面,然后可以在任意节点上看到集群上所有节点的node_info了,所以我们接下来,先建立一个集群环境,然后再在集群环境下尝试做升级。

1.创建2个节点alice bob互连并运行application counter-0.3.0

我们从头开始把counter-0.3.0作为base版本
把counter-0.2.0中的appup.src删掉,并把app.src rebar.config里面的版本号升级一下为0.3.0后打上mnesia-upgrade-0.3.0-alice Tag。
启动alice
脚本文件 ./script/0.3.0-alice-start.sh

#!/bin/bash
git checkout mnesia-upgrade-0.3.0-alice
export RELX_REPLACE_OS_VARS=true
export NODENAME=alice@127.0.0.1
export CLUSTER_NODENAME=bob@127.0.0.1
export COOKIE=zhongwen
rebar3 release
_build/default/rel/counter/bin/counter-0.3.0-alice console

正常启动alice时时结果如下

=PROGRESS REPORT==== 13-Nov-2016::14:32:24 ===
application: sasl
started_at: 'alice@127.0.0.1'
Eshell V7.3 (abort with ^G)
(alice@127.0.0.1)1> [[{node_info,'alice@127.0.0.1',2,true}]]
[[{node_info,'alice@127.0.0.1',3,true}]]
(alice@127.0.0.1)1> application:which_applications().
[{sasl,"SASL CXC 138 11","2.7"},
{counter,"counter","0.3.0"},
{mnesia,"MNESIA CXC 138 12","4.13.3"},
{stdlib,"ERTS CXC 138 10","2.8"},
{kernel,"ERTS CXC 138 10","4.2"}]
(alice@127.0.0.1)2> [[{node_info,'alice@127.0.0.1',4,true}]]
[[{node_info,'alice@127.0.0.1',5,true}]]

基本功能和0.2.0一样,不过会多一个节点互连功能,
把counter-0.3.0-alice的包名改成counter-0.3.0-bob后mnesia-upgrade-0.3.0-bob Tag。
脚本文件 ./script/0.3.0-bob-start.sh

#!/bin/bash
git checkout mnesia-upgrade-0.3.0-bob
export RELX_REPLACE_OS_VARS=true
export NODENAME=bob@127.0.0.1
export CLUSTER_NODENAME=alice@127.0.0.1
export COOKIE=zhongwen
rebar3 release
_build/default/rel/counter/bin/counter-0.3.0-bob console

正常启动bob里输出为

=PROGRESS REPORT==== 13-Nov-2016::14:38:34 ===
application: sasl
started_at: 'bob@127.0.0.1'
Eshell V7.3 (abort with ^G)
(bob@127.0.0.1)1> [[{node_info,'alice@127.0.0.1',113,true}],[{node_info,'bob@127.0.0.1',2,true}]]

可以看到在bob console里面已看到了alice节点的情况
我们再来确认一下alice是否能看到bob

(alice@127.0.0.1)1> [[{node_info,'alice@127.0.0.1',112,true}]]
[[{node_info,'alice@127.0.0.1',113,true}],[{node_info,'bob@127.0.0.1',1,true}]]

2.升级一个为了兼容旧代码的小版本

集群和单节点不一样的是,mnesia table是在alice bob之前共享表结构和数据的,如果我们先升级alice,那么没有升级的bob就会报错,因为bob上跑的是旧代码,他的node_info 没有binary字段,但alice升级就把node_info table升级成带binary字段的table了。这样bob就惨了。
所以我们一定要保证bob在alice升级完成后也不报错,所以在做真正升级时做一个兼容代码的小版本,让旧代码遇到新表数据时也不报错!
我们来做这个兼容小版本叫0.3.1
主要变动就是:

update_count() ->
Fun = fun() ->
- [Client] = mnesia:read(?TABLE, node()),
- mnesia:write(Client#node_info{count = Client#node_info.count + 1,
- smp_support = erlang:system_info(smp_support)})
- end,
+ case mnesia:read(?TABLE, node()) of
+ [Client] when is_record(Client, node_info) ->
+ mnesia:write(Client#node_info{count = Client#node_info.count + 1,
+ smp_support = erlang:system_info(smp_support)
+ });
+ [{node_info, Name, Count, _SmpSupport, _Binary}] ->
+ mnesia:write({node_info, Name, Count + 1, erlang:system_info(smp_support), erlang:memory(binary)})
+ end
+ end,
{atomic, ok} = mnesia:sync_transaction(Fun),
io:format("~p~n", [[mnesia:dirty_read(?TABLE, Key) ||Key <- mnesia:dirty_all_keys(?TABLE)]]).

让这代码可以跑新旧表结果的数据。
所有改动可以使用以下命令看到

git diff mnesia-upgrade-0.3.0-alice mnesia-upgrade-0.3.1-alice

升级alice 0.3.0 到 0.3.1
./script/upgrade-0.3.0-alice-to-0.3.1-alice.sh

#!/bin/bash
git checkout mnesia-upgrade-0.3.1-alice
export RELX_REPLACE_OS_VARS=true
export NODENAME=alice@127.0.0.1
export CLUSTER_NODENAME=bob@127.0.0.1
export COOKIE=zhongwen
rebar3 release
rebar3 relup --upfrom 0.3.0-alice
rebar3 tar
mv _build/default/rel/counter/counter-0.3.1-alice.tar.gz _build/default/rel/counter/releases/0.3.1-alice/counter.tar.gz
_build/default/rel/counter/bin/counter-0.3.0-alice upgrade 0.3.1-alice

成功提示

===> Resolved counter-0.3.1-alice
===> tarball /Users/zhongwen/github/erlang-rock/mneisa-cluster-hot-upgrade-record-structure/counter/_build/default/rel/counter/counter-0.3.1-alice.tar.gz successfully created!
Release 0.3.1-alice not found, attempting to unpack releases/0.3.1-alice/counter.tar.gz
Unpacked successfully: "0.3.1-alice"
Installed Release: 0.3.1-alice
Made release permanent: "0.3.1-alice"

同样升级bob 0.3.0 到 0.3.1
./script/upgrade-0.3.0-bob-to-0.3.1-bob.sh
成功提示

===> Resolved counter-0.3.1-bob
===> tarball /Users/zhongwen/github/erlang-rock/mneisa-cluster-hot-upgrade-record-structure/counter/_build/default/rel/counter/counter-0.3.1-bob.tar.gz successfully created!
Release 0.3.1-bob not found, attempting to unpack releases/0.3.1-bob/counter.tar.gz
Unpacked successfully: "0.3.1-bob"
Installed Release: 0.3.1-bob
Made release permanent: "0.3.1-bob"

3. mnesia node_info table加入binary字段

这里我们从0.3.1升级到0.4.1(加了binary字段)上就和单节点上升级一样操作了,先升级alice,然后再升级bob即可。
主要就是跑以下2个脚本
./script/upgrade-0.3.1-alice-to-0.4.1-alice.sh
./script/upgrade-0.3.1-bob-to-0.4.1-bob.sh
因为步骤一样,所以就不在演示啦。看效果就是
bob alice都在线升级到了0.4.1。可以看到每个节点binary统计。

[[{node_info,'alice@127.0.0.1',90,true,41456}],
[{node_info,'bob@127.0.0.1',87,true,29608}]]

总结

  • mnesia单节点table升级要直接使用mnesia:transform_table/4
  • mnesia集群table升级要先把所有节点的旧代码升级到兼容新旧table的小版本,再依次所有节点从小版本升级到真正需要的版本。