Haisor: Human-aware Indoor Scene Optimization via Deep Reinforcement Learning

3D indoor scene synthesis has been investigated thoroughly, including floorplan generation [Wang et al. 2019; Ritchie et al. 2019; Wang et al. 2018; Liu et al. 2021b] and furniture layout arrangement [Nauata et al. 2020;, 2021; Hu et al. 2020]. For comprehensive discussions on scene synthesis, please refer to survey papers Pintore et al. [2020] and Zhang et al. [2019].

Understanding the furniture arrangements such as their relations is a central topic in scene synthesis. Existing research has made considerable effort to explore approaches that can generate geometrically realistic and reasonable indoor scenes by predicting furniture placement from a semantic perspective and retrieving existing furniture for producing realistic scenes. Various solutions have been proposed by considering different settings, such as example-driven methods [Fisher et al. 2012], language-driven synthesis [Ma et al. 2018], learning-based methods [Merrell et al. 2011; Zhang et al. 2022; Kermani et al. 2016], and optimization-based methods [Xu et al. 2015; Yeh et al. 2012; Henderson and Ferrari 2017]. Recently, deep neural networks have been successfully leveraged to learn the potential distribution from large amounts of data. These works [Wang et al. 2018; Ritchie et al. 2019; Wang et al. 2019] introduce image-based deep generative models to model the placements and relations with the graph of furniture from an enormous amount of top-down view images of scenes. GRAINS [Li et al. 2019] is a pioneering work for scene structure modeling, leveraging hierarchical graphs to represent scenes and producing realistic furniture layouts within a square domain with border walls by Recursive Neural Networks (RvNNs). Moreover, Zhang et al. [2020] presents a hybrid representation to train a scene layout generator through a combination of 2D images and 3D object placements. SceneFormer [Wang et al. 2021b] and ATISS [Paschalidou et al. 2021] predict furniture placement sequentially based on the transformer [Vaswani et al. 2017], which achieve state-of-the-art (SoTA) 3D scene conditional generation. Recently, LayoutEnhancer [Leimer et al. 2022] aims to generate the enhanced scene layout by combining both expert knowledge and a distribution learned from data. Besides that, SceneGraphNet [Zhou et al. 2019] models the short/long-range relations between objects based on a novel neural message-passing network, which can perform object suggestions considering the surrounding objects in a scene.

In contrast to the above scene synthesis frameworks that only consider the realism of synthesized scenes in geometry and layouts, researchers have been exploring other factors of human interaction to ensure the functional plausibility and accessibility of 3D synthesized scenes, e.g., accessibility of objects [Yu et al. 2011], human trajectory [Qi et al. 2018], and human activity [Fu et al. 2017; Ma et al. 2016; Liu et al. 2021a]. Yu et al. [2011] only regard objects as a whole to model the accessibility. Qi et al. [2018] optimize the scene layout according to the learned affordance maps with a stochastic grammar model, without considering the accessibility of furniture (especially for part mobility). Similarly, Ye et al. [2022] synthesize the indoor scene by considering the human motion. Fu et al. [2017] and Ma et al. [2016] optimize the object placement in a scene with pre-defined actions or activity-associated object relation graphs.

Instead, Haisor allows optimizing the furniture arrangements to achieve realism automatically and further enables personalizing 3D scene layout, which accommodates human manipulation with objects and their moving parts to ensure functionality and human affordance for comfortable accessibility.

Rearrangement Planning has been investigated in the robotic field. There is a trend to study how to rearrange objects for the goal configuration using a robot. It is NP-hard [Wilfong 1991] and an interesting challenge area for robotics research. Yuan et al. [2018];, 2019] learn a non-prehensile plan for robots’ arms to place the objects in the right location under some criteria. For multi-object rearrangement planning, a substantial body of works [Haustein et al. 2015; King et al. 2016;, 2017; Koval et al. 2015; Song et al. 2020] has explored this problem in two stages (local path planning and global strategy searching). Bai et al. [2021] proposed a hierarchical policy to solve the non-prehensile multi-object rearrangement with deep reinforcement learning and MCTS. For complex and realistic indoor scenes, SceneMover [Wang et al. 2020] and PEARL [Wang et al. 2021a] solve the scene rearrangement planning problem via selecting pre-defined paths for movement and exploring non-prehensile action space (grasp & pick), respectively. The non-prehensile actions make the solution more tractable.

Prior works in robotics perform algorithms on a robot, which only use some simplified proxies to represent the scene, and it is difficult to describe the full complexity of the scene, e.g., detailed geometry, part mobility, and orientation. Although SceneMover and PEARL conduct the rearrangement planning in realistic scenes, the target is known or learned from the data distribution. In contrast, our method Haisor aims to optimize the furniture placement in realistic scenes with complex part geometry and mobility and further integrates the human-aware factors (affordance and manipulation with objects) for 3D scene layout customization.

The goal of reinforcement learning (RL) [Sutton and Barto 2018] is to help the agent learn good policies for sequential decision problems under dynamic environments by optimizing a cumulative future reward signal in an unsupervised manner. It seeks the gradient directions from countless attempts/trials through facing different incentives from the environment, which is a process of accumulating experience. When the environment is known, it is named model-based RL, which can be solved using dynamic programming. However, most of the real-world problems are environment-unknown, namely, model-free RL, which can be divided into value-based RL and policy-based RL. Policy-based methods are usually combined with a policy gradient algorithm [Achiam et al. 2017; Agarwal et al. 2020; Vuong et al. 2019; Chen et al. 2022]. For value-based methods, Q-Learning is one of the most popular reinforcement learning algorithms. However, it has been observed that it occasionally learns unreasonably high action values, since it includes a maximization process over the estimated action values, which may bias towards overestimation.

With the success of deep neural networks (DNN) in the field of 2D image processing [Krizhevsky et al. 2012], reinforcement learning has seen considerable efforts that use DNNs to fit policy networks, namely, deep reinforcement learning, including DQN [Mnih et al. 2013], Ddouble-DQN [van Hasselt et al. 2016] and Dueling-DQN [Wang et al. 2016]. The deep reinforcement learning algorithms have been investigated and demonstrated their superior performance across many challenging tasks. For example, AlphaZero [Silver et al. 2017] in the games of chess and Go without human knowledge, OpenAI Five [Berner et al. 2019] in the MOBA games for multi-agents cooperation, and learning to self-navigate in complex environments [Mirowski et al. 2017]. Different from the above agents, our agent is performed in the real-world scene layout reconfiguration task. The action space and rewards are designed to optimize the scene layout in a human-aware fashion. Researchers have been exploring numerous approaches to make learning more efficient [Andrychowicz et al. 2016], stable [Mnih et al. 2015], and benefit from experience replay [O’Donoghue et al. 2017]. Levine et al. [2018] learn a policy to control the robots for grasping from the raw inputs of a camera. Scene Mover [Wang et al. 2020] is the most relevant work to ours, which designs a novel algorithm for scene arrangement by automatically generating a move plan. Instead, ours aims to reconfigure the furniture arrangements to realism automatically and further implicitly personalize the 3D scene layout via different reward designs.

Bounding Box vs. Detailed Geometry. In real-world 3D indoor scenes, objects are associated with their detailed geometry. However, the existing scene synthesis methods such as GRAINS [Li et al. 2019] or Deep Priors [Wang et al. 2018] only consider the layout of the scene, using a bounding box to represent the whole furniture. In our setting, object parts are important, because (1) actual human-furniture interaction happens on object parts, and (2) some object placement patterns, e.g., chairs inserted under the table, cannot be modeled using object bounding boxes. Considering the fact that adding detailed geometry of object parts into our framework is too heavy, we choose to use a bounding box to represent each object part. This “hybrid” method is lightweight, and it preserves the object parts’ structure, enabling the two features described above.

Nodes. In our scene graph, every part-level node is associated with an 11-dimensional data of the oriented bounding box of the corresponding part or object, denoted by \([\mathbf {p},\mathbf {e},\mathbf {o},\mathbf {f}]\). \(\mathbf {p}\) and \(\mathbf {e}\) are 3D vectors storing the position and the extent of the bounding box. \(\mathbf {o}\) is a 4D quaternion storing the orientation of the bounding box. \(\mathbf {f}\) is a binary flag, indicating whether the part is movable. In object-level scene graph \((V,E)\), the node data is represented as \([\mathbf {p},\mathbf {e},\mathbf {o},\mathbf {l}]\). \(\mathbf {l}\) is a one-hot vector encoding the category of the object, and its length \(\mathbf {C}\) equals the number of object categories. Thus, the node data of scene graph is a \(|V| \times 11\) matrix for part-level graph and a \(|V| \times (10 \mathbf {C})\) matrix for object-level graph.

Edges. The edges of the graph are designed to hold information of the proximity of nodes. We connect an edge between nodes \(v_1\) and \(v_2\) if the following condition is satisfied:where the \(\Vert \cdot \Vert _2\) means the \(\mathcal{l}_2\) norm. Note that in the part-level graph, an edge between a pair of nodes is connected only if the parents of the two nodes are connected by an edge in the object-level graph. In our scene graph, every edge \((v_1,v_2)\) is associated with a 3D vector \({\bf p}_1 - {\bf p}_2\). Thus, the edge data of a scene graph is an \(|E| \times 3\) matrix.

2-layer Scene Graphs. We need to consider both the relationships between object parts and the relationships between objects. As a solution, the scene graph is extended to a 2-layer one. Two graphs \((V,E)\) and \((V_p,E_p)\) are constructed individually on the object level and part level and associated with the ownership of parts by objects. The ownership is defined as an integer \(a_i\):

Movable parts are essential elements when considering human-scene interaction, while it is more complicated to model them. The two most encountered types (Figure 2) of movable parts are considered in this approach:

Directly feeding the movement data described above into the network is an option, but it is hard for the network to learn the relationship between the data and the actual movement. Instead, we decide to sample three states of the movable parts: the lower limit of movement, the upper limit of movement, and the middle of them. These three bounding boxes are added to the part-level scene graph and treated like other parts. Note that for most movable parts in our data, the lower and upper limit states are either the original state itself, or symmetric to the original state. So, we do not need to include the original bounding boxes.

Walls act as “hard constraints” in our decision sequence. Any part of the furniture must not collide with the walls. Since we do not consider the ceiling and the floor, we can assume the wall is infinitely extended in the up and down directions.

There are many possible representations for walls. We can simply use 3D models or describe the inner boundary of the wall with a 2D ring. However, like in the case of movable parts, directly feeding any type of wall data into the network is not beneficial for learning the relationship between objects and walls. We observe that we need to take into account the object-wall distance to avoid collision. So, we extract the minimum distance between objects and the wall in each direction. Since we do not consider the up and down directions, for each object, we have 4-dimensional data. All the data forms a \(|V| \times 4\) matrix.

Wall-object Distance Extraction. To extract the minimum wall-object distance, we adopt the ray-casting method; the brief diagram is shown in Figure 3. Due to the complicated representations of walls in the 3D-FRONT dataset [Fu et al. 2021] (e.g., walls represented by separate meshes with numerous vertices, instead of regular shapes), it is hard to calculate the distance between the wall and the bounding box of objects, and the simple 2D closed-form method is not available. Given an object bounding box and a direction, we first decide on the corresponding face. For example, if the direction is \( x\), then we select the front face of the bounding box. We then sample 10 points equidistantly along the other axis (in the example y-axis), with the center of the selected face in the middle. Then, for each point, we construct a ray. The origins of the rays are the sampled points, and the directions are the input direction. We test the interaction of these rays and the wall. The minimum distance among these intersections is our final result.

Human-object interaction. We use motion planning on a humanoid robot to imitate the interaction between human and movable parts of objects. We simulate the interaction in three steps:

We illustrate our procedure to produce human-object interaction information in Figure 4. Additionally, more sophisticated reward settings based on motion planning results can be added. With these extended settings, joint values of motion planning can be considered to determine the quality of human-object interaction. Please refer to the supplementary material for details.

Free space of human activity. We assume that a human occupies a volume of the shape of an elliptic cylinder (1,800mm \(\times\) 750mm \(\times\) 550mm) according to the dimensions of a typical human body [Panero and Zelnik 1999]. By testing all the possible positions that a person can stand, we extract the amount of free space for human activity. We compute the amount of free space in two steps:

We illustrate our procedure to produce free-space information in Figure 5.

To train the Q-Network described in Section 3, we employ reinforcement learning techniques. For each scene in the training dataset, we create a virtual environment. The agent explores the environment using the \(\epsilon\)-greedy policy, and each action taken by the agent results in an immediate reward. Once the agent stops optimizing, it is given a final reward that indicates the overall “quality” of the optimized scene. The states and rewards are stored in a replay buffer, from which actual training data is sampled. We provide details of the settings in this section.

Action Space. The action taken by the agent can be written as a pair \((v, d)\), where v is a node in the object-level graph \((V,E)\), and d is a move action type (or direction).

There are four types of actions \(t_{ x},t_{-x},t_{ z},t_{-z}\). Every action refers to moving the object towards a direction denoted by its subscript for 0.1 m, and they can be indexed by 0,1,2,3. We encode all actions in a scene into a number \(a \in \mathbb {N}\). If the action is to move object i towards direction d, then we have

Rewards. When the agent decides to take a specific action, the state of the simulation environment is changed according to the action, and an immediate reward is given to the agent when every step of action is taken. Additionally, a final reward is given when the agent decides to stop the optimization. An immediate reward is a form of reward shaping, since the final reward is too sparse for the agent to learn. The components of the reward are described below.

Stopping. Unlike the episodic task in Wang et al. [2020], our reinforcement learning task is a continuous task without a goal or stopping state. The only “episodic” task in our settings is to eliminate all collisions between furniture, thus, we give the first-time collision elimination reward to encourage the reduction of collisions. To tackle the stopping problem, we feed the average feature of all the nodes into an MLP and output a single Q-value indicating the reward obtained by immediately stopping the optimization. To fit the stopping action into our action-simulation pipeline, we add a “virtual object” after the actual objects, extending our action space dimension from \(4 \times |V|\) to \(4 \times (|V| 1)\). The Q-value of all the four actions of the virtual object equals to the predicted Q-value. Once the virtual object is selected by the agent, we stop the simulation, concluding the optimization process.

The goal of the Q-network is to predict the Q-value \(Q(s,a)\) for state s and action a. For any state s, the optimal action predicted by the network is \(\max \nolimits _{a^{\prime }}Q(s,a^{\prime })\).

Network Structure. The Q-network is a Dueling Double Deep Q-Network based on a Graph Convolution Network (GCN). The input of the network is a scene graph representation \(\lbrace (V,E), (V_p,E_p)\rbrace\), and the output of the network is the estimated Q-value \(\hat{Q}(s,a)\) for every possible action a.

The first building block of our network is GCN. For an arbitrary input graph \((V,E)\), the GCN performs multiple message-passing passes on the graph, and every message-passing pass can be described aswhere \(V_i^{(k)}\) is the feature of the ith node after k rounds of message passing, \(N_i\) is the number of neighbors of the ith node, \(E_{ij}\) is the feature attached to the edge \((i,j)\), and \(f^{(k)}\) is a learnable encoder.

We first perform several rounds of message passing on the part-level scene graph \((V_p,E_p)\) and obtain a matrix of data \(V_p^{(k)}\) of shape \(|V_p| \times L_{feature}\), where \(L_{feature}\) is the length of feature vector output.

The second step is to aggregate the features of part-level nodes that are owned by the same object. This can be denoted bywhere \(\delta (i,j)\) is 1 when \(i=j\), and 0 otherwise.

After the aggregation of part-level node features, we then concatenate object-level feature \(V_i\) and the distance of object i to the walls in all four directions to the corresponding row of the feature matrix, changing shape of the feature matrix to \(|V| \times (L_{feature} 11 4)\). Then, we perform another round of message passing on this feature matrix and edges of object-level graph E. The obtained data \(V^{(k)}\) is fed into two networkswhere \(f_{state}\) and \(f_{action}\) are Multi-Layer Perceptrons (MLPs). We use this two-branch network structure after Dueling Q-Network [Wang et al. 2016]. \(Q_s\) is a scalar representing the Q-value of the state, and \(Q_{i}^{\prime }\) is a 4-dimensional vector representing the Q-value of every action performed on object i.

The following equation is used to produce the final estimated Q-value \(\hat{Q}(s,a)\):where \(i(a) = \lfloor (a \div 4)\rfloor , d(a) = a\mod {4}\), \(x[n]\) is to obtain the data of vector x on index n.

Double Q-network. According to DQN [van Hasselt et al. 2016], using a single Q-network in both action selection and action evaluation may cause the overestimation of Q-values. To solve this problem, we use two networks of the same structure \(Q_1(s,a),Q_2(s,a)\). When training, the target Q-value is computed usingwhere y is the target Q-value, and r is the immediate reward given by taking the action.

Wall Constraints. Walls act as hard constraints, but our Q-network is not capable of handling the constraints directly, so we set the Q-value of the invalid actions that will produce new collisions between objects and walls to \(-\infty\) in the output layer of our Q-network.

To optimize the training of our network, we adopt the prioritized replay buffer and imitation learning techniques and further extend the imitation learning to a version that can be bootstrapped by learning the policies of a simple heuristics agent at the beginning of training.

Prioritized Replay Buffer. In the training process of a deep Q-network, a set of data \((s_t,a,s_{t 1},r_t)\) is needed, where \(s_t,s_{t 1}\) denotes the states in time t and \(t 1\), a is the action taken, and \(r_t\) is the immediate reward. During exploration, every step produces a set of data, and the data is stored in a buffer called Replay Buffer. Since our rationality and human affordance reward is sparse, only a small amount of data in the buffer is needed to correctly learn the Q-value. Thus, we associate every set of data with a priority \(p = |y - Q(s_t,a)|\). This priority makes the data that the current Q-network predicts incorrectly to be sampled more frequently [Schaul et al. 2016]. The sampling algorithm is further controlled by parameters \(\alpha\) and \(\beta\).

Imitation Learning. The main problem for our network training lies in the slow speed of the initial exploration of the \(\epsilon\)-greedy agent. Overall, our environment has sparse rewards, which means positive rewards can only be obtained in a few states. If we use the \(\epsilon\)-greedy strategy, then the agent will spend a lot of time searching for positive rewards. Thus, we consider generating a sequence of actions known as demonstration data, which can achieve a subset of our goals, e.g., collision elimination. Since the demonstration data is only used in the early stage of training, it can be sub-optimal. This training strategy is called imitation training [Hester et al. 2018]. The demonstration data is considered in the training by adding a loss termwhere \(a_d\) is the action in demonstration data, and c is a positive constant. This loss term forces the Q-value of \(a_d\) to be higher than other actions by c.

The demonstration data can be obtained by multiple methods. The most straightforward method is human annotation. However, we can produce the demonstration data by a simple heuristic method consisting of several rules:

Monte Carlo Tree Search is a heuristic algorithm widely used in AI agents of board games. In our setting, the algorithm constructs a tree, the nodes of which are states of the environment. Each node is associated with a value, representing the expectation of the reward gained by selecting the best actions starting from this state. To decide an action for a certain state of the environment, the algorithm performs multiple iterations. Every iteration is divided into four steps:

Training Data. To our knowledge, 3D indoor scene datasets with movable parts are unavailable. The 3D-FRONT [Fu et al. 2021] is a currently available 3D indoor scene dataset that is created by professional designers. We take the scenes generated by the underlying generators (see the next section) and replace the 3D-FUTURE [Fu et al. 2021] meshes with PartNet [Mo et al. 2019] and PartNet-mobility [Xiang et al. 2020] meshes. For each 3D-FUTURE mesh, we retrieve top-10 meshes with the smallest Chamfer Distance [Barrow et al. 1977] from PartNet. The 3D-FUTURE mesh in the scene is randomly replaced with top-10 retrieved meshes.

Note that this simple mesh-replacement approach may produce a large number of scenes that contain objects with inappropriate movable parts. For example, nearly half of the scenes have cabinet doors that cannot be opened. Since most indoor scene generation pipelines choose furniture using the scales of the bounding box of furniture, our replaced furniture can simulate the generated scenes by such methods.

Scene Generators. To test our optimization framework, we utilize two SoTA scene generators: Sync2Gen [Yang et al. 2021] and ATISS [Paschalidou et al. 2021]. They are both trained on the 3D-FRONT dataset and generate layouts with the 3D box representation. Following these two works, we then retrieve meshes based on the sizes of generated boxes from the PartNet dataset. The generated scenes are then fed into different scene optimization algorithms to be evaluated. Considering the scale of the scenes to be optimized, the maximum optimization step count of the random, heuristic, and Haisor agents in the following experiments are limited to 80 steps.

In our experiments, we perform the comparison with three agents, including the random agent, heuristics agent, and Sync2Gen-Opt agent:

Besides, to demonstrate the changes relative to the originally generated scenes, the metrics measured on the originally generated scenes are also reported.

To evaluate the performance quantitatively, we adopt three kinds of metrics: accuracy, collision, and human affordance [Qi et al. 2018].

Accuracy. To measure the overall realism of the optimized scenes, a ResNet18 [He et al. 2016] is trained to regress the score for each top-down rendered image of the generated scene. The training data of the network consists of top-down renderings of randomly perturbed 3D-FRONT scenes and scores between 0 and 1. The scores are assigned as follows: First, we define a constant distance \(d_{max}\) for every type of room, which indicates the maximum distance of furniture from its original position. The distance is 0.35 m for bedrooms and 1.00 m for living rooms. Second, we pick a random value between 0 and 1, denoted \(d_{rand}\). Third, we move every furniture in a random direction along the ground for a distance of \(d_{rand} \cdot d_{max}\). The score of the scene is then assigned as \(1 - d_{rand}\).

The rendered images have \(2 \mathbf {C}\) channels of three types: (a) Floor channel: takes the value of 255 when inside the floor; (b) Layout channel: takes the value of 255 when inside any furniture object; (c) Category channels: a channel for each object type and takes the value of 255 when inside any furniture object of that category. Any other pixel is assigned the value of 0.

Note that both the reward given by the simulation environment and the “accuracy” reported by experiments below make use of this network, but we use different sets of 3D-FRONT scene data to train two versions of the network in practice. We randomly split the 3D-FRONT scenes into two datasets with equal sizes and train the two networks of the same architecture separately.

Collision. The collision metric measures the collisions between different objects and between objects and the wall. It is calculated as \(N_{coll} = 3 \times N_{wall} N_{obj}\), where \(N_{wall}\) is the number of collisions between objects and the wall, and \(N_{obj}\) is the number of collisions between different objects. Note that \(N_{wall}\) and \(N_{obj}\) are both collision numbers, which means any small collision between objects will count as 1 collision.

Human Affordance. According to the discussion of human-scene interaction in Section 3, we evaluate two metrics associated with human affordance: (1) Movable Manipulation, defined as the percentage p described in Section 5.1, and (2) Free Space, defined as the free space of human activity divided by available space, which is exactly \(R_{fs} \div 15\) in Section 5.1. For the detailed calculation of the two metrics, please refer to Section 5.1. The overall metric of human affordance can be regarded as the average of these two individual metrics.

The comparison of scene optimization. We perform optimization on the generated scenes by every baseline, namely, Sync2Gen [Yang et al. 2021] and ATISS [Paschalidou et al. 2021]. In Table 1, we show the numerical results of different baselines on scene optimization of generated scenes. According to the generated scene, we can see that both ATISS and Sync2Gen-VAE commonly perform worse in the living rooms than in bedrooms, which indicates the generation in living room is more challenging than in bedrooms. After the optimization, our method performs the best on all three metrics, indicating that Haisor is able to reconfigure the furniture placement to satisfy the designed criteria. In Figure 7, some optimized visual results are presented for different baselines. For the generated results of living rooms, more collisions between furniture still exist and more movable parts cannot be manipulated. We observe that our method captures the “functional region” of a set of furniture better, and our method can achieve multiple goals (e.g., solving collisions, extending free space, and finding space for movable parts) simultaneously, while the heuristic agent is only capable of solving collisions, Sync2Gen-Opt agent struggles to strike a balance between multiple goals, and the simulated annealing agent tends to overly separate the objects apart and fails to combine furniture into meaningful regions.

Furthermore, in our optimization results, we observed some smart strategies. One of the examples is shown in Figure 8. In this example, the initial scene is a living room, in which six chairs are placed together, causing a lot of collisions. The agent first moves two chairs of the same size to the center of the room, forming a region of “chairs surrounding a table” and subsequently moves four other chairs, forming the second region of “chairs surrounding a table.” Finally, the agent makes some additional moves that increase the human affordance of the scene. We observe that our agent can learn the key feature for a scene to be realistic (in this case, the “chair-table” region) and derive the corresponding actions. Examples of more strategies learned to improve rationality and human affordance can be found in the supplementary material. Additionally, we include another example in Figure 9. The initial scene is overall rational, with the chair-table region and the table-sofa region placed together correctly. However, there are three objects with movable parts: two hinges of cabinets and one slider of a table. There is not enough space for humans to manipulate them comfortably. The results in Figure 9 demonstrate that our pipeline is able to optimize the scene to get more space for movable parts while preserving the rationality of the scene.

Perceptual Study. In addition to quantitative and qualitative results, we conduct a perceptual study to further evaluate how realistic and viable the optimized results of our method and two baseline methods (Sync2Gen-Opt and Heuristics agent) are. To fairly compare the results, we randomly select 50 living rooms and 50 bedrooms generated by Sync2Gen-VAE and ATISS to be optimized. We render the scenes in a top-down view and ask each participant to rank each algorithm according to two criteria (Rationality and Human Affordance): (a) the overall performance of the optimized rooms; (b) whether the participants will enjoy themselves when living in the scene. In every round of the survey, the participants are presented with two scenes for each method, including the input scene before optimization (without informing participants which is which) and rank the five results according to the above criteria. All the participants were local volunteers known to be reliable. The results of this perceptual study are reported in Table 2, and our method is the most preferred among the five results (the input scene and three optimized results). Further, we can see that the input scene gets a high score due to users’ preference for the overall rationality of the scene, and Sync2Gen-Opt and heuristic agents may move furniture for long distances and make the scene look messy, although they may have higher metrics of the collision or human affordance.

Ablation Study. The performance of our method relies heavily on various components of rewards used in the simulation environment, the training methodology (i.e., imitation learning), and the Monte Carlo Tree Search. We perform ablation studies to demonstrate the key designs of our method.

We compare our full method with eight ablated versions of our method, where we remove a certain design of our method for each ablated version and train the network on the same settings as our full method. Note that we have included the w/o GCN ablated version, which replaces the GCN in the Q-network with a plain MLP. The MLP is simply fed with the concatenated feature as input, and the dimension of the output is \(4 \times V_{max}\), where \(V_{max}\) is the maximum number of furniture in a room. But, the first \(4 \times |V|\) elements of the output are used as the predicted Q-value, where \(|V|\) is the real number of furniture of a certain scene. The rest elements are simply discarded in our experiments. We have also included a “Pure MCTS” ablated version of the agent, which makes decisions solely based on the search results of MCTS without the Q-Network prior.

The quantitative metrics are reported in Table 3. Statistics from the table clearly indicate that our full model outperforms all the ablated versions overall. The ablated methods achieve similar performance to our full method on some metrics (e.g., collision or human affordance), but our method is capable of considering all the aspects simultaneously. The qualitative results of all ablated versions and ours are presented in Figure 10. It is very clear that the results without imitation learning are totally worse than our full model under the same coverage steps. The agent without MCTS fails to move the chair under the table. When the free space and human affordance components are removed from our model, the furniture is moved to be highly scattered, which results in not enough space for humans to manipulate the cabinet beside the dining table and chairs. Without the collision reward, there are still some unresolved slight collisions, since they have almost no effect on rationality and human affordance. Without the rationality reward, the agent behaves similarly to the heuristics agent, which only tends to resolve collision. If the network is replaced by a plain MLP, then the agent is only capable of capturing part of the relationships between objects but fails to perform as well as the agent with GCN.

In addition to the rearrangement of the scene layout, our method can be extended for scene customization by adding more components to the reward settings. The motivation of our extension is to solve some common issues of general generative models. There are two key observations: (1) some of the generative models still predict the unreasonable orientation of objects. For example, it is unrealistic that a chair faces the wall. (2) Some objects are duplicated and lie in the same location, such as two dining chairs surrounding the dining table are predicted at similar locations.

Based on the observation above, we perform two types of extensions: orientation adjustment and object removal. In general, we extend the action space for each object by adding an extra action below:

Figures 11 and 12 show two examples of the extensions of our method. In Figure 11, one dining chair around the dining table does not face the table, while three other chairs do. This is frequently seen in the generation results of the SoTA scene generative models. The predicted orientation is often aligned with the x or z axes, so we only need to adjust the orientation by \(\pi /2, \pi , 3\pi /2\) radians or rotate the object by \(\pi /2\) for 1,2,3 times. In Figure 12, two dining tables are predicted in exactly the same position. This is very common in the current generative models based on sequential generation, such as some frameworks based on transformers. To optimize this scene, we only need to remove one of the two tables to make the indoor scene more realistic.

The two examples above are extensions of our optimization method. Since our method is based on a simulation environment, Reinforcement Learning, and MCTS, Haisor is able to achieve various optimization actions and goals. By simply adding some personalized actions or rewards, our method can optimize the indoor scene to the different targets that satisfy individual needs.