目录
- 概述
- 源码解析
- 1. main部分
- 1.1 SecondStageMain-第二阶段启动
- 1.2 MountExtraFilesystems-挂载apex和linkerconfig目录到tmpfs中
- 1.3 InstallSignalFdHandler-信号处理相关的
- 1.4 HandleSignalFd-信号处理函数
- 1.5 InstallInitNotifier-初始化eventfd，相当于发信号唤醒init进程的
- 1.6 WakeMainInitThread-唤醒init进程
- 1.7 RecordStageBoottimes-设置init，first_stage，selinux的ro.boottime属性
- 1.8 export_oem_lock_status-根据androidboot.verifiedbootstate设置ro.boot.flash.locked
- 1.9 LoadBootScripts-解析rc文件，初始化actionmanager和servicelist
- 1.10 CreateParser-创建Parser对象
- 1.11 HandleProcessActions-服务重启相关
- 2. util类
- 3. Epoll类
- 4. MountHandler类
- 5. subcontext模块
- 6. mount namespace模块
- 1. main部分
- 问题
- 补充
- 参考
概述
-
创建进程会话密钥
-
初始化属性服务-创建线程-建立socket通信-用来设置属性的
-
SELinux 第二阶段准备后续工作
-
初始化子进程终止信号处理函数
-
初始化init进程唤醒函数-使用eventfd机制-唤醒来处理action
-
设置mount namespace
-
新建subcontext进程-建立socket通信-让subcontext执行命令
-
init.rc 文件解析 & 处理
源码解析
1. main部分
1.1 SecondStageMain-第二阶段启动
int SecondStageMain(int argc, char** argv) {
// 异常之后，重启相关的
if (REBOOT_BOOTLOADER_ON_PANIC) {
InstallRebootSignalHandlers();
}
boot_clock::time_point start_time = boot_clock::now();
// 关机回调
trigger_shutdown = [](const std::string& command) { shutdown_state.TriggerShutdown(command); };
// 将标准输出设置为/dev/null
SetStdioToDevNull(argv);
// 初始化内核打印
InitKernelLogging(argv);
LOG(INFO) << "init second stage started!";
// Init should not crash because of a dependence on any other process, therefore we ignore
// SIGPIPE and handle EPIPE at the call site directly. Note that setting a signal to SIG_IGN
// is inherited across exec, but custom signal handlers are not. Since we do not want to
// ignore SIGPIPE for child processes, we set a no-op function for the signal handler instead.
{// 处理SIGPIPE信号
struct sigaction action = {.sa_flags = SA_RESTART};
action.sa_handler = [](int) {}; // 就什么都不做
sigaction(SIGPIPE, &action, nullptr);
}
// Set init and its forked children's oom_adj.
if (auto result = // 往oom_score_adj里面写-1000，表示不能杀掉
WriteFile("/proc/1/oom_score_adj", StringPrintf("%d", DEFAULT_OOM_SCORE_ADJUST));
!result.ok()) {
LOG(ERROR) << "Unable to write " << DEFAULT_OOM_SCORE_ADJUST
<< " to /proc/1/oom_score_adj: " << result.error();
}
// Set up a session keyring that all processes will have access to. It
// will hold things like FBE encryption keys. No process should override
// its session keyring.
// 新建进程会话密钥环，其中
// - KEYCTL_GET_KEYRING_ID：表示通过第二个参数的类型获取当前进程的密钥信息
// - KEY_SPEC_SESSION_KEYRING：表示获取当前进程的 SESSION_KEYRING
// - 1：表示如果获取不到就新建一个会话密钥环
keyctl_get_keyring_ID(KEY_SPEC_SESSION_KEYRING, 1);
// Indicate that booting is in progress to background fw loaders, etc.
close(open("/dev/.booting", O_WRONLY | O_CREAT | O_CLOEXEC, 0000));
// See if need to load debug props to allow adb root, when the device is unlocked.
// 这个属性会在first_stage_init阶段设置
const char* force_debuggable_env = getenv("INIT_FORCE_DEBUGGABLE");
bool load_debug_prop = false;
// unlock是判断cmdline中androidboot.verifiedbootstate=orange
if (force_debuggable_env && AvbHandle::IsDeviceUnlocked()) {
load_debug_prop = "true"s == force_debuggable_env;
}
unsetenv("INIT_FORCE_DEBUGGABLE");
// Umount the debug ramdisk so property service doesn't read .prop files from there, when it
// is not meant to.
// 不unmountdebugramdisk的话，就可以读到debug相关的属性，adb_debug.prop和userdebug_plat_sepolicy.cil文件
if (!load_debug_prop) {
UmountDebugRamdisk();
}
// 初始化属性相关的东西
PropertyInit();
// Umount the debug ramdisk after property service has read the .prop files when it means to.
if (load_debug_prop) {
UmountDebugRamdisk();
}
// Mount extra filesystems required during second stage init
// 挂载apex和linkerconfig的文件系统，因为根目录是只读的，所以需要把apex和linkerconfig挂载到tmpfs才能创建文件夹
MountExtraFilesystems();
// Now set up SELinux for second stage.
SelinuxSetupKernelLogging();
SelabelInitialize();
SelinuxRestoreContext();
Epoll epoll;
if (auto result = epoll.Open(); !result.ok()) {
PLOG(FATAL) << result.error();
}
// 信号处理函数
InstallSignalFdHandler(&epoll);
// 用来唤醒init进程的
InstallInitNotifier(&epoll);
// 创建property_service有名socket，创建property_fd 无名socket（用来和init进程通信），以及propertyservice线程
StartPropertyService(&property_fd);
// Make the time that init stages started available for bootstat to log.
// 设置init，first_stage，selinux的ro.boottime属性
RecordStageBoottimes(start_time);
// Set libavb version for Framework-only OTA match in Treble build.
// 设置avb的版本号
if (const char* avb_version = getenv("INIT_AVB_VERSION"); avb_version != nullptr) {
SetProperty("ro.boot.avb_version", avb_version);
}
unsetenv("INIT_AVB_VERSION");
// 挂载vendor_overlay，没有vendor_overlay目录
fs_mgr_vendor_overlay_mount_all();
// 根据androidboot.verifiedbootstate设置ro.boot.flash.locked
export_oem_lock_status();
// 根据/proc/mounts文件来设置mount相关的属性，dev.mnt.blk开头
MountHandler mount_handler(&epoll);
// 根据/sys/class/udc设置sys.usb.controller属性
SetUsbController();
// 返回builtins的函数
const BuiltinFunctionMap& function_map = GetBuiltinFunctionMap();
// 设置action的builtins的函数，static变量
Action::set_function_map(&function_map);
// 设置mount namespace相关的东西，根目录下默认全部都是shared的
if (!SetupMountNamespaces()) {
PLOG(FATAL) << "SetupMountNamespaces failed";
}
// 初始化subcontext
InitializeSubcontext();
// 创建ActionManager对象，ServiceList对象
ActionManager& am = ActionManager::GetInstance();
ServiceList& sm = ServiceList::GetInstance();
// 加载rc文件，保存到actionmanager和servicelist中
LoadBootScripts(am, sm);
// Turning this on and letting the INFO logging be discarded adds 0.2s to
// Nexus 9 boot time, so it's disabled by default.
if (false) DumpState();
// Make the GSI status available before scripts start running.
// gsi相关的
auto is_running = android::gsi::IsGsiRunning() ? "1" : "0";
SetProperty(gsi::kGsiBootedProp, is_running);
auto is_installed = android::gsi::IsGsiInstalled() ? "1" : "0";
SetProperty(gsi::kGsiInstalledProp, is_installed);
// 构建action，放到event_queue，等待执行函数
am.QueueBuiltinAction(SetupCgroupsAction, "SetupCgroups");
am.QueueBuiltinAction(SetKptrRestrictAction, "SetKptrRestrict");
am.QueueBuiltinAction(TestPerfEventSelinuxAction, "TestPerfEventSelinux");
am.QueueEventTrigger("early-init");
// Queue an action that waits for coldboot done so we know ueventd has set up all of /dev...
am.QueueBuiltinAction(wait_for_coldboot_done_action, "wait_for_coldboot_done");
// ... so that we can start queuing up actions that require stuff from /dev.
am.QueueBuiltinAction(MixHwrngIntoLinuxRngAction, "MixHwrngIntoLinuxRng");
am.QueueBuiltinAction(SetMmapRndBitsAction, "SetMmapRndBits");
Keychords keychords;
am.QueueBuiltinAction(
[&epoll, &keychords](const BuiltinArguments& args) -> Result<void> {
for (const auto& svc : ServiceList::GetInstance()) {
keychords.Register(svc->keycodes());
}
keychords.Start(&epoll, HandleKeychord);
return {};
},
"KeychordInit");
// Trigger all the boot actions to get us started.
am.QueueEventTrigger("init");
// Repeat mix_hwrng_into_linux_rng in case /dev/hw_random or /dev/random
// wasn't ready immediately after wait_for_coldboot_done
am.QueueBuiltinAction(MixHwrngIntoLinuxRngAction, "MixHwrngIntoLinuxRng");
// Don't mount filesystems or start core system services in charger mode.
std::string bootmode = GetProperty("ro.bootmode", "");
if (bootmode == "charger") {
am.QueueEventTrigger("charger");
} else {
am.QueueEventTrigger("late-init");
}
// Run all property triggers based on current state of the properties.
am.QueueBuiltinAction(queue_property_triggers_action, "queue_property_triggers");
while (true) {
// By default, sleep until something happens.
auto epoll_timeout = std::optional<std::chrono::milliseconds>{};
// 关机相关
auto shutdown_command = shutdown_state.CheckShutdown();
if (shutdown_command) {
HandlePowerctlMessage(*shutdown_command);
}
// 第一次默认为false
if (!(prop_waiter_state.MightBeWaiting() || Service::is_exec_service_running())) {
// 执行action
am.ExecuteOneCommand();
}
if (!IsShuttingDown()) {
// 服务重启相关
auto next_process_action_time = HandleProcessActions();
// If there's a process that needs restarting, wake up in time for that.
if (next_process_action_time) {
epoll_timeout = std::chrono::ceil<std::chrono::milliseconds>(
*next_process_action_time - boot_clock::now());
if (*epoll_timeout < 0ms) epoll_timeout = 0ms;
}
}
// exec执行是要waiting的；persist属性的设置初始化，要等待
if (!(prop_waiter_state.MightBeWaiting() || Service::is_exec_service_running())) {
// If there's more work to do, wake up again immediately.
// timeout为0
if (am.HasMoreCommands()) epoll_timeout = 0ms;
}
// 没有事件，就立刻返回
auto pending_functions = epoll.Wait(epoll_timeout);
if (!pending_functions.ok()) {
LOG(ERROR) << pending_functions.error();
// signal，mount_handle，notification唤醒，keycode相关，property_service，subcontext相关
} else if (!pending_functions->empty()) {
// We always reap children before responding to the other pending functions. This is to
// prevent a race where other daemons see that a service has exited and ask init to
// start it again via ctl.start before init has reaped it.
ReapAnyOutstandingChildren();
// 执行epoll的functions
for (const auto& function : *pending_functions) {
(*function)();
}
}
if (!IsShuttingDown()) {
// 处理属性的control命令，ctl.开头的属性
HandleControlMessages();
// 设置usb属性，sys.usb.controller，只需要设置一次
SetUsbController();
}
}
return 0;
}
1.2 MountExtraFilesystems-挂载apex和linkerconfig目录到tmpfs中
static void MountExtraFilesystems() {
#define CHECKCALL(x) \
if ((x) != 0) PLOG(FATAL) << #x " failed.";
// /apex is used to mount APEXes
CHECKCALL(mount("tmpfs", "/apex", "tmpfs", MS_NOEXEC | MS_NOSUID | MS_NODEV,
"mode=0755,uid=0,gid=0"));
// /linkerconfig is used to keep generated linker configuration
CHECKCALL(mount("tmpfs", "/linkerconfig", "tmpfs", MS_NOEXEC | MS_NOSUID | MS_NODEV,
"mode=0755,uid=0,gid=0"));
#undef CHECKCALL
}
1.3 InstallSignalFdHandler-信号处理相关的
static void InstallSignalFdHandler(Epoll* epoll) {
// Applying SA_NOCLDSTOP to a defaulted SIGCHLD handler prevents the signalfd from receiving
// SIGCHLD when a child process stops or continues (b/77867680#comment9).
// SIG_DFL使用默认的信号处理函数，SA_NOCLDSTOP就是child进程被杀之后，不接受其发来的SIGCHLD信号
const struct sigaction act { .sa_handler = SIG_DFL, .sa_flags = SA_NOCLDSTOP };
// 设置SIGCHLD信号的信号处理函数，和signal函数一样
sigaction(SIGCHLD, &act, nullptr);
sigset_t mask;
sigemptyset(&mask);
// 添加SIGCHLD信号
sigaddset(&mask, SIGCHLD);
if (!IsRebootCapable()) {
// If init does not have the CAP_SYS_BOOT capability, it is running in a container.
// In that case, receiving SIGTERM will cause the system to shut down.
sigaddset(&mask, SIGTERM);
}
// SIG_BLOCK：mask = mask|set 添加屏蔽字
// SIG_UNBLOCK：mask = mask|~set 解除屏蔽字
// SIG_SETMASK：mask = set 赋值操作
if (sigprocmask(SIG_BLOCK, &mask, nullptr) == -1) {
PLOG(FATAL) << "failed to block signals";
}
// Register a handler to unblock signals in the child processes.
// int pthread_atfork(void (*prepare)(void), void (*parent)(void), void (*child)(void));
// prepare是fork之前执行，parent是fork之后在父进程执行，child是fork之后在子进程执行
// 将子进程的信号处理恢复原样
const int result = pthread_atfork(nullptr, nullptr, &UnblockSignals);
if (result != 0) {
LOG(FATAL) << "Failed to register a fork handler: " << strerror(result);
}
signal_fd = signalfd(-1, &mask, SFD_CLOEXEC);
if (signal_fd == -1) {
PLOG(FATAL) << "failed to create signalfd";
}
if (auto result = epoll->RegisterHandler(signal_fd, HandleSignalFd); !result.ok()) {
LOG(FATAL) << result.error();
}
}
// 将子进程的信号处理恢复原样
static void UnblockSignals() {
const struct sigaction act { .sa_handler = SIG_DFL };
sigaction(SIGCHLD, &act, nullptr);
sigset_t mask;
sigemptyset(&mask);
sigaddset(&mask, SIGCHLD);
sigaddset(&mask, SIGTERM);
if (sigprocmask(SIG_UNBLOCK, &mask, nullptr) == -1) {
PLOG(FATAL) << "failed to unblock signals for PID " << getpid();
}
}
1.4 HandleSignalFd-信号处理函数
static void HandleSignalFd() {
signalfd_siginfo siginfo;
ssize_t bytes_read = TEMP_FAILURE_RETRY(read(signal_fd, &siginfo, sizeof(siginfo)));
if (bytes_read != sizeof(siginfo)) {
PLOG(ERROR) << "Failed to read siginfo from signal_fd";
return;
}
switch (siginfo.ssi_signo) {
case SIGCHLD:
ReapAnyOutstandingChildren();
break;
case SIGTERM:
HandleSigtermSignal(siginfo);
break;
default:
PLOG(ERROR) << "signal_fd: received unexpected signal " << siginfo.ssi_signo;
break;
}
}
1.5 InstallInitNotifier-初始化eventfd，相当于发信号唤醒init进程的
// 相当于给init进程发信号
static int wake_main_thread_fd = -1;
static void InstallInitNotifier(Epoll* epoll) {
// 创建一个eventfd
wake_main_thread_fd = eventfd(0, EFD_CLOEXEC);
if (wake_main_thread_fd == -1) {
PLOG(FATAL) << "Failed to create eventfd for waking init";
}
auto clear_eventfd = [] {
uint64_t counter;
// 就读一下，其实啥都没做
TEMP_FAILURE_RETRY(read(wake_main_thread_fd, &counter, sizeof(counter)));
};
if (auto result = epoll->RegisterHandler(wake_main_thread_fd, clear_eventfd); !result.ok()) {
LOG(FATAL) << result.error();
}
}
1.6 WakeMainInitThread-唤醒init进程
static void WakeMainInitThread() {
uint64_t counter = 1;
// 往里写，就唤醒init进程了
TEMP_FAILURE_RETRY(write(wake_main_thread_fd, &counter, sizeof(counter)));
}
1.7 RecordStageBoottimes-设置init，first_stage，selinux的ro.boottime属性
static void RecordStageBoottimes(const boot_clock::time_point& second_stage_start_time) {
int64_t first_stage_start_time_ns = -1;
if (auto first_stage_start_time_str = getenv(kEnvFirstStageStartedAt);
first_stage_start_time_str) {
SetProperty("ro.boottime.init", first_stage_start_time_str);
android::base::ParseInt(first_stage_start_time_str, &first_stage_start_time_ns);
}
unsetenv(kEnvFirstStageStartedAt);
int64_t selinux_start_time_ns = -1;
if (auto selinux_start_time_str = getenv(kEnvSelinuxStartedAt); selinux_start_time_str) {
android::base::ParseInt(selinux_start_time_str, &selinux_start_time_ns);
}
unsetenv(kEnvSelinuxStartedAt);
if (selinux_start_time_ns == -1) return;
if (first_stage_start_time_ns == -1) return;
SetProperty("ro.boottime.init.first_stage",
std::to_string(selinux_start_time_ns - first_stage_start_time_ns));
SetProperty("ro.boottime.init.selinux",
std::to_string(second_stage_start_time.time_since_epoch().count() -
selinux_start_time_ns));
}
1.8 export_oem_lock_status-根据androidboot.verifiedbootstate设置ro.boot.flash.locked
static void export_oem_lock_status() {
if (!android::base::GetBoolProperty("ro.oem_unlock_supported", false)) {
return;
}
ImportKernelCmdline([](const std::string& key, const std::string& value) {
if (key == "androidboot.verifiedbootstate") {
SetProperty("ro.boot.flash.locked", value == "orange" ? "0" : "1");
}
});
}
1.9 LoadBootScripts-解析rc文件，初始化actionmanager和servicelist
static void LoadBootScripts(ActionManager& action_manager, ServiceList& service_list) {
Parser parser = CreateParser(action_manager, service_list);
std::string bootscript = GetProperty("ro.boot.init_rc", "");
if (bootscript.empty()) {
parser.ParseConfig("/system/etc/init/hw/init.rc");
if (!parser.ParseConfig("/system/etc/init")) {
late_import_paths.emplace_back("/system/etc/init");
}
// late_import is available only in Q and earlier release. As we don't
// have system_ext in those versions, skip late_import for system_ext.
parser.ParseConfig("/system_ext/etc/init");
if (!parser.ParseConfig("/product/etc/init")) {
late_import_paths.emplace_back("/product/etc/init");
}
if (!parser.ParseConfig("/odm/etc/init")) {
late_import_paths.emplace_back("/odm/etc/init");
}
if (!parser.ParseConfig("/vendor/etc/init")) {
late_import_paths.emplace_back("/vendor/etc/init");
}
} else {
parser.ParseConfig(bootscript);
}
}
1.10 CreateParser-创建Parser对象
// 由下面可以看到：
// rc文件就三个关键字打头：service on import
Parser CreateParser(ActionManager& action_manager, ServiceList& service_list) {
// 创建parser对象，什么都没做
Parser parser;
// 存储三个对象指针
parser.AddSectionParser("service", std::make_unique<ServiceParser>(
&service_list, GetSubcontext(), std::nullopt));
parser.AddSectionParser("on", std::make_unique<ActionParser>(&action_manager, GetSubcontext()));
parser.AddSectionParser("import", std::make_unique<ImportParser>(&parser));
return parser;
}
1.11 HandleProcessActions-服务重启相关
static std::optional<boot_clock::time_point> HandleProcessActions() {
std::optional<boot_clock::time_point> next_process_action_time;
for (const auto& s : ServiceList::GetInstance()) {
if ((s->flags() & SVC_RUNNING) && s->timeout_period()) {
auto timeout_time = s->time_started() + *s->timeout_period();
if (boot_clock::now() > timeout_time) {
s->Timeout();
} else {
if (!next_process_action_time || timeout_time < *next_process_action_time) {
next_process_action_time = timeout_time;
}
}
}
if (!(s->flags() & SVC_RESTARTING)) continue;
auto restart_time = s->time_started() + s->restart_period();
if (boot_clock::now() > restart_time) {
if (auto result = s->Start(); !result.ok()) {
LOG(ERROR) << "Could not restart process '" << s->name() << "': " << result.error();
}
} else {
if (!next_process_action_time || restart_time < *next_process_action_time) {
next_process_action_time = restart_time;
}
}
}
return next_process_action_time;
}
2. util类
2.1 IsLegalPropertyName-小于1，以.开头，最后一个为.，以及包含..都是不行的
bool IsLegalPropertyName(const std::string& name) {
size_t namelen = name.size();
if (namelen < 1) return false;
if (name[0] == '.') return false;
if (name[namelen - 1] == '.') return false;
/* Only allow alphanumeric, plus '.', '-', '@', ':', or '_' */
/* Don't allow ".." to appear in a property name */
for (size_t i = 0; i < namelen; i++) {
if (name[i] == '.') {
// i=0 is guaranteed to never have a dot. See above.
if (name[i - 1] == '.') return false;
continue;
}
if (name[i] == '_' || name[i] == '-' || name[i] == '@' || name[i] == ':') continue;
if (name[i] >= 'a' && name[i] <= 'z') continue;
if (name[i] >= 'A' && name[i] <= 'Z') continue;
if (name[i] >= '0' && name[i] <= '9') continue;
return false;
}
return true;
}
2.2 IsLegalPropertyValue-判断是否有无效的多字节字符
Result<void> IsLegalPropertyValue(const std::string& name, const std::string& value) {
// ro.开头的就可以无限制了？
if (value.size() >= PROP_VALUE_MAX && !StartsWith(name, "ro.")) {
return Error() << "Property value too long";
}
// 判断是否有无效的多字节字符
if (mbstowcs(nullptr, value.data(), 0) == static_cast<std::size_t>(-1)) {
return Error() << "Value is not a UTF8 encoded string";
}
return {};
}
2.3 ImportKernelCmdline-解析内核cmdline
void ImportKernelCmdline(const std::function<void(const std::string&, const std::string&)>& fn) {
std::string cmdline;
android::base::ReadFileToString("/proc/cmdline", &cmdline);
// 空格分，=分开来
for (const auto& entry : android::base::Split(android::base::Trim(cmdline), " ")) {
std::vector<std::string> pieces = android::base::Split(entry, "=");
if (pieces.size() == 2) {
fn(pieces[0], pieces[1]);
}
}
}
2.4 CreateSocket-创建socket
// PROP_SERVICE_NAME, SOCK_STREAM | SOCK_CLOEXEC | SOCK_NONBLOCK, false, 0666, 0, 0, {}
// property_service(bionic)
Result<int> CreateSocket(const std::string& name, int type, bool passcred, mode_t perm, uid_t uid,
gid_t gid, const std::string& socketcon) {
// 设置socket的selinux context
if (!socketcon.empty()) {
if (setsockcreatecon(socketcon.c_str()) == -1) {
return ErrnoError() << "setsockcreatecon(\"" << socketcon << "\") failed";
}
}
// 创建socket
android::base::unique_fd fd(socket(PF_UNIX, type, 0));
if (fd < 0) {
return ErrnoError() << "Failed to open socket '" << name << "'";
}
if (!socketcon.empty()) setsockcreatecon(nullptr);
struct sockaddr_un addr;
memset(&addr, 0 , sizeof(addr));
addr.sun_family = AF_UNIX;
// 设置路径为：/dev/socket/property_service
snprintf(addr.sun_path, sizeof(addr.sun_path), ANDROID_SOCKET_DIR "/%s", name.c_str());
if ((unlink(addr.sun_path) != 0) && (errno != ENOENT)) {
return ErrnoError() << "Failed to unlink old socket '" << name << "'";
}
std::string secontext;
if (SelabelLookupFileContext(addr.sun_path, S_IFSOCK, &secontext) && !secontext.empty()) {
setfscreatecon(secontext.c_str());
}
if (passcred) {
int on = 1;
if (setsockopt(fd, SOL_SOCKET, SO_PASSCRED, &on, sizeof(on))) {
return ErrnoError() << "Failed to set SO_PASSCRED '" << name << "'";
}
}
int ret = bind(fd, (struct sockaddr *) &addr, sizeof (addr));
int savederrno = errno;
if (!secontext.empty()) {
setfscreatecon(nullptr);
}
// make_scope_guard相当于下面的函数执行错误，然后goto out，删除socket文件
auto guard = android::base::make_scope_guard([&addr] { unlink(addr.sun_path); });
if (ret) {
errno = savederrno;
return ErrnoError() << "Failed to bind socket '" << name << "'";
}
// 改变socket文件的uid和gid
if (lchown(addr.sun_path, uid, gid)) {
return ErrnoError() << "Failed to lchown socket '" << addr.sun_path << "'";
}// 改变socket文件的mode
if (fchmodat(AT_FDCWD, addr.sun_path, perm, AT_SYMLINK_NOFOLLOW)) {
return ErrnoError() << "Failed to fchmodat socket '" << addr.sun_path << "'";
}
LOG(INFO) << "Created socket '" << addr.sun_path << "'"
<< ", mode " << std::oct << perm << std::dec
<< ", user " << uid
<< ", group " << gid;
guard.Disable();
return fd.release();
}
2.5 ExpandProps-解析import rc中的属性
// /system/etc/init/hw/init.${ro.zygote}.rc
Result<std::string> ExpandProps(const std::string& src) {
const char* src_ptr = src.c_str();
std::string dst;
/* - variables can either be $x.y or ${x.y}, in case they are only part
* of the string.
* - will accept $$ as a literal $.
* - no nested property expansion, i.e. ${foo.${bar}} is not supported,
* bad things will happen
* - ${x.y:-default} will return default value if property empty.
*/
while (*src_ptr) {
const char* c;
c = strchr(src_ptr, '$');
if (!c) {
// 没有$了，返回/system/etc/init/hw/init.zygote32.rc
dst.append(src_ptr);
return dst;
}
// /system/etc/init/hw/init.
dst.append(src_ptr, c);
c++;
// c={，所以不走这里
if (*c == '$') {
dst.push_back(*(c++));
src_ptr = c;
continue;
} else if (*c == '\0') {
return dst;
}
std::string prop_name;
std::string def_val;
// 走这里
if (*c == '{') {
c++;
const char* end = strchr(c, '}');
if (!end) {
// failed to find closing brace, abort.
return Error() << "unexpected end of string in '" << src << "', looking for }";
}
prop_name = std::string(c, end);
c = end + 1;
size_t def = prop_name.find(":-");
if (def < prop_name.size()) {
//走这里
def_val = prop_name.substr(def + 2);
prop_name = prop_name.substr(0, def);
}
} else {
prop_name = c;
if (SelinuxGetVendorAndroidVersion() >= __ANDROID_API_R__) {
return Error() << "using deprecated syntax for specifying property '" << c
<< "', use ${name} instead";
} else {
LOG(ERROR) << "using deprecated syntax for specifying property '" << c
<< "', use ${name} instead";
}
c += prop_name.size();
}
if (prop_name.empty()) {
return Error() << "invalid zero-length property name in '" << src << "'";
}
// 获取属性
std::string prop_val = android::base::GetProperty(prop_name, "");
if (prop_val.empty()) {
if (def_val.empty()) {
return Error() << "property '" << prop_name << "' doesn't exist while expanding '"
<< src << "'";
}
prop_val = def_val;
}
// /system/etc/init/hw/init.zygote32
dst.append(prop_val);
// 再走一次循环
src_ptr = c;
}
return dst;
}
2.6 DecodeUid-可以传名字和uid数字
Result<uid_t> DecodeUid(const std::string& name) {
if (isalpha(name[0])) {
// 从名字解析
passwd* pwd = getpwnam(name.c_str());
if (!pwd) return ErrnoError() << "getpwnam failed";
return pwd->pw_uid;
}
errno = 0;
// 直接返回数字
uid_t result = static_cast<uid_t>(strtoul(name.c_str(), 0, 0));
if (errno) return ErrnoError() << "strtoul failed";
return result;
}
3. Epoll类
3.1 Open-epoll_create1创建epoll
Result<void> Epoll::Open() {
if (epoll_fd_ >= 0) return {};
epoll_fd_.reset(epoll_create1(EPOLL_CLOEXEC));
if (epoll_fd_ == -1) {
return ErrnoError() << "epoll_create1 failed";
}
return {};
}
3.2 RegisterHandler-设置处理函数
Result<void> Epoll::RegisterHandler(int fd, std::function<void()> handler, uint32_t events) {
// events默认是EPOLLIN，读
if (!events) {
return Error() << "Must specify events";
}
auto [it, inserted] = epoll_handlers_.emplace(fd, std::move(handler));
if (!inserted) {
return Error() << "Cannot specify two epoll handlers for a given FD";
}
epoll_event ev;
ev.events = events;
// std::map's iterators do not get invalidated until erased, so we use the
// pointer to the std::function in the map directly for epoll_ctl.
// 信号处理函数指针
ev.data.ptr = reinterpret_cast<void*>(&it->second);
if (epoll_ctl(epoll_fd_, EPOLL_CTL_ADD, fd, &ev) == -1) {
Result<void> result = ErrnoError() << "epoll_ctl failed to add fd";
epoll_handlers_.erase(fd);
return result;
}
return {};
}
3.3 Wait-等待事件来处理
Result<std::vector<std::function<void()>*>> Epoll::Wait(
std::optional<std::chrono::milliseconds> timeout) {
int timeout_ms = -1;
if (timeout && timeout->count() < INT_MAX) {
timeout_ms = timeout->count();
}
const auto max_events = epoll_handlers_.size();
epoll_event ev[max_events];
auto num_events = TEMP_FAILURE_RETRY(epoll_wait(epoll_fd_, ev, max_events, timeout_ms));
if (num_events == -1) {
return ErrnoError() << "epoll_wait failed";
}
std::vector<std::function<void()>*> pending_functions;
for (int i = 0; i < num_events; ++i) {
pending_functions.emplace_back(reinterpret_cast<std::function<void()>*>(ev[i].data.ptr));
}
return pending_functions;
}
4. MountHandler类
4.1 MountHandler-监听/proc/mounts文件
MountHandler::MountHandler(Epoll* epoll) : epoll_(epoll), fp_(fopen("/proc/mounts", "re"), fclose) {
if (!fp_) PLOG(FATAL) << "Could not open /proc/mounts";
auto result = epoll->RegisterHandler(
// 处理mounts文件
fileno(fp_.get()), [this]() { this->MountHandlerFunction(); }, EPOLLERR | EPOLLPRI);
if (!result.ok()) LOG(FATAL) << result.error();
}
4.2 MountHandlerFunction
void MountHandler::MountHandlerFunction() {
rewind(fp_.get());
std::vector<MountHandlerEntry> touched;
// 初始值
auto untouched = mounts_;
char* buf = nullptr;
size_t len = 0;
while (getline(&buf, &len, fp_.get()) != -1) {
auto entry = ParseMount(std::string(buf));
auto match = untouched.find(entry);
// 找不到的，就加进去
if (match == untouched.end()) {
touched.emplace_back(std::move(entry));
} else {
// 本来就存在的，就去掉
untouched.erase(match);
}
}
free(buf);
// 去除的
for (auto& entry : untouched) {
SetMountProperty(entry, false);
mounts_.erase(entry);
}// 新加的
for (auto& entry : touched) {
SetMountProperty(entry, true);
mounts_.emplace(std::move(entry));
}
}
4.3 ParseMount
MountHandlerEntry ParseMount(const std::string& line) {
auto fields = android::base::Split(line, " ");
while (fields.size() < 3) fields.emplace_back("");
// 没有/dev/root的
if (fields[0] == "/dev/root") {
auto& dm = dm::DeviceMapper::Instance();
std::string path;
if (dm.GetDmDevicePathByName("system", &path) || dm.GetDmDevicePathByName("vroot", &path)) {
fields[0] = path;
} else if (android::fs_mgr::Fstab fstab; android::fs_mgr::ReadDefaultFstab(&fstab)) {
auto entry = GetEntryForMountPoint(&fstab, "/");
if (entry || (entry = GetEntryForMountPoint(&fstab, "/system"))) {
fields[0] = entry->blk_device;
}
}
}
// 凡是/dev/开头的
if (android::base::StartsWith(fields[0], "/dev/")) {
// 真实的路径
if (std::string link; android::base::Readlink(fields[0], &link)) {
fields[0] = link;
}
}
// 返回MountHandlerEntry，blk_device(blk_device), mount_point(mount_point), fs_type(fs_type)
return MountHandlerEntry(fields[0], fields[1], fields[2]);
}
4.4 SetMountProperty-设置dev.mnt.blk属性
void SetMountProperty(const MountHandlerEntry& entry, bool add) {
static constexpr char devblock[] = "/dev/block/";
// 要以/dev/block/开头的
if (!android::base::StartsWith(entry.blk_device, devblock)) return;
std::string value;
if (add) {
value = entry.blk_device.substr(strlen(devblock));
if (android::base::StartsWith(value, "sd")) {
// All sd partitions inherit their queue characteristics
// from the whole device reference. Strip partition number.
auto it = std::find_if(value.begin(), value.end(), [](char c) { return isdigit(c); });
if (it != value.end()) value.erase(it, value.end());
}
auto queue = "/sys/block/" + value + "/queue";
struct stat sb;
// 文件夹要存在，/sys/block/dm-4/queue文件夹和挂载点的文件夹
if (stat(queue.c_str(), &sb) || !S_ISDIR(sb.st_mode)) value = "";
if (stat(entry.mount_point.c_str(), &sb) || !S_ISDIR(sb.st_mode)) value = "";
// Clear the noise associated with loopback and APEX.
// 去掉loop和apex
if (android::base::StartsWith(value, "loop")) value = "";
if (android::base::StartsWith(entry.mount_point, "/apex/")) value = "";
}
auto mount_prop = entry.mount_point;
if (mount_prop == "/") mount_prop = "/root";
std::replace(mount_prop.begin(), mount_prop.end(), '/', '.');
mount_prop = "dev.mnt.blk" + mount_prop;
// Set property even if its value does not change to trigger 'on property:'
// handling, except for clearing non-existent or already clear property.
// Goal is reduction of empty properties and associated triggers.
if (value.empty() && android::base::GetProperty(mount_prop, "").empty()) return;
android::base::SetProperty(mount_prop, value);
}
5. subcontext模块
用来隔离system和vendor，system的context是u:r:init:s0的，vendor的context是u:r:vendor_init:s0的。
通过socket来和init进程进行通信，接收init进程发来的命令，然后使用u:r:vendor_init:s0 context来执行命令。
vendor和odm的rc文件，也是使用u:r:vendor_init:s0来执行的，还有vendor的一些属性
5.1 InitializeSubcontext-初始化subcontext
void InitializeSubcontext() {
if (SelinuxGetVendorAndroidVersion() >= __ANDROID_API_P__) {
subcontext.reset(// u:r:vendor_init:s0
new Subcontext(std::vector<std::string>{"/vendor", "/odm"}, kVendorContext));
}
}
5.2 Subcontext构造函数-新建subcontext进程ps -Af可以看到
Subcontext(std::vector<std::string> path_prefixes, std::string context)
: path_prefixes_(std::move(path_prefixes)), context_(std::move(context)), pid_(0) {
// 新建进程
Fork();
}
5.3 Fork-新建subcontext进程
void Subcontext::Fork() {
unique_fd subcontext_socket;
if (!Socketpair(AF_UNIX, SOCK_SEQPACKET | SOCK_CLOEXEC, 0, &socket_, &subcontext_socket)) {
LOG(FATAL) << "Could not create socket pair to communicate to subcontext";
return;
}
auto result = fork();
if (result == -1) {
LOG(FATAL) << "Could not fork subcontext";
} else if (result == 0) {
socket_.reset();
// We explicitly do not use O_CLOEXEC here, such that we can reference this FD by number
// in the subcontext process after we exec.
// 新的fd，指向同一个文件
int child_fd = dup(subcontext_socket); // NOLINT(android-cloexec-dup)
if (child_fd < 0) {
PLOG(FATAL) << "Could not dup child_fd";
}
// We don't switch contexts if we're running the unit tests. We don't use std::optional,
// since we still need a real context string to pass to the builtin functions.
if (context_ != kTestContext) {
// 切换context
if (setexeccon(context_.c_str()) < 0) {
PLOG(FATAL) << "Could not set execcon for '" << context_ << "'";
}
}
auto init_path = GetExecutablePath();
auto child_fd_string = std::to_string(child_fd);
const char* args[] = {init_path.c_str(), "subcontext", context_.c_str(),
child_fd_string.c_str(), nullptr};
// 执行init subcontext u:r:vendor_init:s0 15
// 执行SubcontextMain函数
execv(init_path.data(), const_cast<char**>(args));
PLOG(FATAL) << "Could not execv subcontext init";
} else {
subcontext_socket.reset();
pid_ = result;
LOG(INFO) << "Forked subcontext for '" << context_ << "' with pid " << pid_;
}
}
5.4 SubcontextMain-subcontext进程
int SubcontextMain(int argc, char** argv, const BuiltinFunctionMap* function_map) {
if (argc < 4) LOG(FATAL) << "Fewer than 4 args specified to subcontext (" << argc << ")";
auto context = std::string(argv[2]);
auto init_fd = std::atoi(argv[3]);
SelabelInitialize();
trigger_shutdown = [](const std::string& command) { shutdown_command = command; };
auto subcontext_process = SubcontextProcess(function_map, context, init_fd);
//进程干活的地方
subcontext_process.MainLoop();
return 0;
}
5.5 MainLoop
void SubcontextProcess::MainLoop() {
pollfd ufd[1];
ufd[0].events = POLLIN;
ufd[0].fd = init_fd_;
while (true) {
ufd[0].revents = 0;
// 监听sockets
int nr = TEMP_FAILURE_RETRY(poll(ufd, arraysize(ufd), -1));
if (nr == 0) continue;
if (nr < 0) {
PLOG(FATAL) << "poll() of subcontext socket failed, continuing";
}
// 从sockets读消息
auto init_message = ReadMessage(init_fd_);
if (!init_message.ok()) {
if (init_message.error().code() == 0) {
// If the init file descriptor was closed, let's exit quietly. If
// this was accidental, init will restart us. If init died, this
// avoids calling abort(3) unnecessarily.
return;
}
LOG(FATAL) << "Could not read message from init: " << init_message.error();
}
auto subcontext_command = SubcontextCommand();
// 解析sockets的消息
if (!subcontext_command.ParseFromString(*init_message)) {
LOG(FATAL) << "Unable to parse message from init";
}
auto reply = SubcontextReply();
switch (subcontext_command.command_case()) {
case SubcontextCommand::kExecuteCommand: {
// 执行命令
RunCommand(subcontext_command.execute_command(), &reply);
break;
}
case SubcontextCommand::kExpandArgsCommand: {
ExpandArgs(subcontext_command.expand_args_command(), &reply);
break;
}
default:
LOG(FATAL) << "Unknown message type from init: "
<< subcontext_command.command_case();
}
if (!shutdown_command.empty()) {
reply.set_trigger_shutdown(shutdown_command);
shutdown_command.clear();
}
// 返回执行命令的结果给init进程
if (auto result = SendMessage(init_fd_, reply); !result.ok()) {
LOG(FATAL) << "Failed to send message to init: " << result.error();
}
}
}
6. mount namespace模块
mount namespace是相对于进程来说的，每个进程都有自己的mount namespace
1. 查看进程的mount namespace是不是一样的
ls -alh /proc/pid/ns/
2. 进程建立自己独立的mount namespace
unshare(CLONE_NEWNS)
在通过CLONE_NEWNS创建mount namespace后，父进程会把自己的文件结构复制给子进程中。而子进程中新的namespace中的所有mount操作都只影响自身的文件系统，而不对外界产生任何影响。这样可以做到比较严格地隔离。
拷贝之后，增删互不影响
6.1 SetupMountNamespaces
bool SetupMountNamespaces() {
// Set the propagation type of / as shared so that any mounting event (e.g.
// /data) is by default visible to all processes. When private mounting is
// needed for /foo/bar, then we will make /foo/bar as a mount point (by
// bind-mounting by to itself) and set the propagation type of the mount
// point to private.
// 默认设根目录为shared，对所有的mount namespace都可见
if (!MakeShared("/", true /*recursive*/)) return false;
// /apex is a private mountpoint to give different sets of APEXes for
// the bootstrap and default mount namespaces. The processes running with
// the bootstrap namespace get APEXes from the read-only partition.
// 设/apex为private的，不可见的
if (!(MakePrivate("/apex"))) return false;
// /linkerconfig is a private mountpoint to give a different linker configuration
// based on the mount namespace. Subdirectory will be bind-mounted based on current mount
// namespace
// /linkerconfig也为不可见的
if (!(MakePrivate("/linkerconfig"))) return false;
// The two mount namespaces present challenges for scoped storage, because
// vold, which is responsible for most of the mounting, lives in the
// bootstrap mount namespace, whereas most other daemons and all apps live
// in the default namespace. Scoped storage has a need for a
// /mnt/installer view that is a slave bind mount of /mnt/user - in other
// words, all mounts under /mnt/user should automatically show up under
// /mnt/installer. However, additional mounts done under /mnt/installer
// should not propagate back to /mnt/user. In a single mount namespace
// this is easy to achieve, by simply marking the /mnt/installer a slave
// bind mount. Unfortunately, if /mnt/installer is only created and
// bind mounted after the two namespaces are created below, we end up
// with the following situation:
// /mnt/user and /mnt/installer share the same peer group in both the
// bootstrap and default namespaces. Marking /mnt/installer slave in either
// namespace means that it won't propagate events to the /mnt/installer in
// the other namespace, which is still something we require - vold is the
// one doing the mounting under /mnt/installer, and those mounts should
// show up in the default namespace as well.
//
// The simplest solution is to do the bind mount before the two namespaces
// are created: the effect is that in both namespaces, /mnt/installer is a
// slave to the /mnt/user mount, and at the same time /mnt/installer in the
// bootstrap namespace shares a peer group with /mnt/installer in the
// default namespace.
// /mnt/androidwritable is similar to /mnt/installer but serves for
// MOUNT_EXTERNAL_ANDROID_WRITABLE apps.
if (!mkdir_recursive("/mnt/user", 0755)) return false;
if (!mkdir_recursive("/mnt/installer", 0755)) return false;
if (!mkdir_recursive("/mnt/androidwritable", 0755)) return false;
if (!(BindMount("/mnt/user", "/mnt/installer", true))) return false;
if (!(BindMount("/mnt/user", "/mnt/androidwritable", true))) return false;
// First, make /mnt/installer and /mnt/androidwritable a slave bind mount
if (!(MakeSlave("/mnt/installer"))) return false;
if (!(MakeSlave("/mnt/androidwritable"))) return false;
// Then, make it shared again - effectively creating a new peer group, that
// will be inherited by new mount namespaces.
if (!(MakeShared("/mnt/installer"))) return false;
if (!(MakeShared("/mnt/androidwritable"))) return false;
// 保存bootstrap的ns
bootstrap_ns_fd.reset(OpenMountNamespace());
bootstrap_ns_id = GetMountNamespaceId();
// When APEXes are updatable (e.g. not-flattened), we create separate mount
// namespaces for processes that are started before and after the APEX is
// activated by apexd. In the namespace for pre-apexd processes, small
// number of essential APEXes (e.g. com.android.runtime) are activated.
// In the namespace for post-apexd processes, all APEXes are activated.
bool success = true;
if (IsApexUpdatable() && !IsRecoveryMode()) {
// Creating a new namespace by cloning, saving, and switching back to
// the original namespace.
// pre-apex是创建新的mount namespace
if (unshare(CLONE_NEWNS) == -1) {
PLOG(ERROR) << "Cannot create mount namespace";
return false;
}
default_ns_fd.reset(OpenMountNamespace());
default_ns_id = GetMountNamespaceId();
if (setns(bootstrap_ns_fd.get(), CLONE_NEWNS) == -1) {
PLOG(ERROR) << "Cannot switch back to bootstrap mount namespace";
return false;
}
} else {
// Otherwise, default == bootstrap
default_ns_fd.reset(OpenMountNamespace());
default_ns_id = GetMountNamespaceId();
}
// /system/apex下只有文件，没有文件夹，所以这个函数啥都没干
success &= ActivateFlattenedApexesIfPossible();
LOG(INFO) << "SetupMountNamespaces done";
return success;
}
6.2 mount-类型MS_BIND，MS_SHARED，MS_SLAVE，MS_PRIVATE
// MS_BIND：使该挂载点在另一个挂载点上可视
// MS_SHARED：挂载和卸载的消息在各个mount namespace共享
// MS_PRIVATE：挂载点不向任何对等方传播事件，也不从任何对等方接收传播事件。在mount namespace之间不共享
// MS_SLAVE：其成员将挂载和卸载事件传播到从属挂载。但是，从属挂载不会将事件传播到主对等组。
// cat /proc/self/mountinfo可以看到master和shared
static bool BindMount(const std::string& source, const std::string& mount_point,
bool recursive = false) {
unsigned long mountflags = MS_BIND;
if (recursive) {
mountflags |= MS_REC;
}
if (mount(source.c_str(), mount_point.c_str(), nullptr, mountflags, nullptr) == -1) {
PLOG(ERROR) << "Failed to bind mount " << source;
return false;
}
return true;
}
static bool MakeShared(const std::string& mount_point, bool recursive = false) {
unsigned long mountflags = MS_SHARED;
if (recursive) {
mountflags |= MS_REC;
}
if (mount(nullptr, mount_point.c_str(), nullptr, mountflags, nullptr) == -1) {
PLOG(ERROR) << "Failed to change propagation type to shared";
return false;
}
return true;
}
static bool MakeSlave(const std::string& mount_point, bool recursive = false) {
unsigned long mountflags = MS_SLAVE;
if (recursive) {
mountflags |= MS_REC;
}
if (mount(nullptr, mount_point.c_str(), nullptr, mountflags, nullptr) == -1) {
PLOG(ERROR) << "Failed to change propagation type to slave";
return false;
}
return true;
}
static bool MakePrivate(const std::string& mount_point, bool recursive = false) {
unsigned long mountflags = MS_PRIVATE;
if (recursive) {
mountflags |= MS_REC;
}
if (mount(nullptr, mount_point.c_str(), nullptr, mountflags, nullptr) == -1) {
PLOG(ERROR) << "Failed to change propagation type to private";
return false;
}
return true;
}
6.3 IsApexUpdatable-ro.apex.updatable属性
static bool IsApexUpdatable() {
// 这个定义在android/system/apex/apexd/sysprop/ApexProperties.sysprop文件中
// 也就是ro.apex.updatable属性
static bool updatable = android::sysprop::ApexProperties::updatable().value_or(false);
return updatable;
}
问题
串口remount失败问题
补充
1. epoll
它的功能是监视多个文件描述符以查看是否可以在其中任何一个上进行 I/O
// 创建epoll
int epoll_create1(int flags);
int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event);
int epoll_wait(int epfd, struct epoll_event *events, int maxevents, int timeout);
触发方式：
edge边缘触发：epoll_wait只有在新数据写入管道时才会返回。如果只读了部分数据，那么剩下的数据就没办法读到了。
level电平触发：epoll_wait只要管道的缓冲区包含要读取的数据，进一步调用将立即返回。
2. std::set自定义类型，find比较
// 相等的情况：two keys k1 and k2 are considered to be equivalent if for the comparison object comp, comp(k1, k2) == false && comp(k2, k1) == false.
bool MountHandlerEntry::operator<(const MountHandlerEntry& r) const {
if (blk_device < r.blk_device) return true;
if (blk_device > r.blk_device) return false;
if (mount_point < r.mount_point) return true;
if (mount_point > r.mount_point) return false;
return fs_type < r.fs_type;
}
// 找，是否有这个字符串
auto match = untouched.find(entry);
// 找不到的
if (match == untouched.end()) {
touched.emplace_back(std::move(entry));
} else {
// 找到了
untouched.erase(match);
}
3. std::make_unique-构造指针
// 先创建ActionParser对象，然后获取其指针
std::make_unique<ActionParser>(&action_manager, GetSubcontext())
参考
（1）Linux内核安全模块学习-内核密钥管理子系统
https://blog.csdn.net/asmartkiller/article/details/108611798
（2）android 8.1 安全机制 — SEAndroid & SELinux
https://blog.csdn.net/qq_19923217/article/details/81240027
（3）Android 8.1 开机流程分析（2）
https://blog.csdn.net/qq_19923217/article/details/82014989
（4）Linux 密钥保留服务keyring入门
https://blog.csdn.net/chengfenglee/article/details/112666506
（5）Linux进程间通信（一）： 信号 signal()、sigaction()
https://www.cnblogs.com/52php/p/5813867.html
（6）信号集 & sigprocmask、sigpending
https://blog.csdn.net/weixin_36750623/article/details/83061785
（7）std::set、自定义类型与比较函数
https://www.cnblogs.com/shawnhue/archive/2011/12/22/set_comparison_strick_weak_ordering.html
（8）Mount namespaces and shared subtrees
https://lwn.net/Articles/689856/
原文链接: https://www.cnblogs.com/pyjetson/p/14901076.html
欢迎关注
微信关注下方公众号，第一时间获取干货硬货；公众号内回复【pdf】免费获取数百本计算机经典书籍
原创文章受到原创版权保护。转载请注明出处：https://www.ccppcoding.com/archives/211499
非原创文章文中已经注明原地址，如有侵权，联系删除
关注公众号【高性能架构探索】，第一时间获取最新文章
转载文章受原作者版权保护。转载请注明原作者出处！
