Task 2-computational creativity, problem space search, optimization, systems view, default network, simulated annealing

Question 1

computational creativity

Answer 1

Computational Creativity (CC) – emerging branch of AI that studies and exploits the potential of computers to be more than feature-rich tools, and to act as autonomous creators and co-creators in their own right

• Computers which generate outputs for an external user to evaluate are merely generative in their behavior, and mere generation does not rise to the level of human creativity•
must exhibit an ability to filter its outputs for quality, so that any outputs presented to a user show intentionality and discernment, and it must exhibit an ability to articulate why its outputs may have interesting and unexpected value for its audience

Question 2

four different criteria for categorizing an answer to a question, or a solution, as creative (Newell, Shaw and Simon, 1983)

Answer 2

four different criteria for categorizing an answer to a question, or a solution, as creative (Newell, Shaw and Simon, 1983)

1. The answer has novelty and usefulness (for the individual or for society)
2. The answer demands that we reject ideas we had previously accepted
3. The answer results from intense motivation and persistence
4. The answer comes from clarifying a problem that was originally vague

Question 3

pastiche

Answer 3

mere appearance of creativity is due to some specifiable slice of the programmer’s own creativity having been imprinted onto the algorithmic workings of the system  mimics rather than creates/ no innovation

Question 4

Exploratory creativity

Answer 4

explores the space as it is defined by the problem, looking for previously undiscovered or unappreciated states of unexpectedly high value

Shortest path can sometimes lead to failure

searching conceptual space defined by implicit rules
AARON: painting robot
Has a specific ‚thinking style‘ = easier for AI

Question 5

Transformation creativity

Answer 5

actively transforms the space

Surprises

Question 6

small c

Answer 6

everyday creativity on a mundane scale

Question 7

big C

Answer 7

exemplary creativity on a historical scale

Question 8

Components of a neural network:

Answer 8

1. Set of processing units or nodes that are similar to neurons but not as complicated
2. A state of activation. The one or two most activated nodes correspond to whatever is the focus of attention. Less activated nodes constitute the contents of STM.
3. A pattern of connections among the nodes which can be excitatory or inhibitory. They constitute LTM.
4. Input and output rules concerning how a node adds up its input and how outputs relate to current activation
5. Learning rules (Hebbian learning)
6. An environment for the network. The networks should be partitioned into modules devoted to specific tasks. Each molecule is organized into several layers with vertical connections being excitatory and lateral inhibition operating on each layer
   • Cognition is massively parallel; all the nodes do whatever they do at the same time
    This is quite unlike a conventional computer which can only do one thing at a time

Question 9

Creative Ideas

Answer 9

Creative Ideas
• Useful and novel
• Useful: appropriate for the domain in which the idea occurs

Question 10

The Creative Process (Helmholtz, 1896)

Answer 10

The Creative Process (Helmholtz, 1896)

1. Preparation: work intensively with a problem without coming to a solution, ideas related to the problem are learned and manipulated in an intellectual fashion
2. Incubation: no progress is happening, so the problem is set aside
3. Illumination: the solution occurred with no clear cause, usually ideas that were not thought to be relevant provided the insight
4. Verification: intellectual scrutiny of the idea

Question 11

associative hierarchies

Answer 11

• When attention is focused, a few nodes are highly activated and exert strong lateral inhibition on other nodes to prevent them to become activated  associated hierarchy is steep
• When attention is less focused, more nodes will be activated but to a lesser degree  associative hierarchy will be relatively flat: the person acts in a more variable fashion and thus is more creative

Question 12

Primary process thinking and creativity

Answer 12

• Kris: creative people are more able to alternate between primary process and secondary process cognition
• Primary process thinking – analogical, autistic, and free associative  makes the discovery of new combinations more likely, corresponds to a state in which large numbers of nodes are weakly activated
• Secondary process – abstract, logical, goal oriented, reality oriented  the most extreme form is deductive reasoning: no creative insight possible because the conclusion is implicit in the premises, corresponds to focused attention
• a creative person may have fantasies or reveries about, say, prime numbers
• Secondary process thinking is best modeled as a state of focused attention, where a few nodes are strongly activated

Question 13

creativity and arousal

Answer 13

• Lower arousal  more nodes become activated  thinking moves from secondary process to primary process
• Creative people show more extreme spontaneous fluctuations in level of arousal

Question 14

simulated annealing

Answer 14

 simulated annealing:
1. Start out with high temperature: nodes are going on and off randomly  can increase and decrease their contribution to total energy which allows the system to crawl out of local minima
2. Lower temperature: by the time the system freezes, it is likely to be at the global minimum
• A network that anneals periodically (oscillating between high and low temperature): when temperature is high one could say the network is working in a primary fashion and when temperature is low, it is operating in a secondary fashion
 the biological analogue of temperature is the inverse for cortical arousal  high temperature corresponds to low arousal

Question 15

Creativity (GRUNER)

Answer 15

• Unique, original, novel + useful
• Deep learning machine programs are now capable of adapting to unexpected situations (self-driving cars)
• Machines are bound to utilitarian algorithms; make decisions without affective and cognitive restraints
1. Combinational creativity
2. Exploratory creativity
3. Transformational creativity

Question 16

computational creativity

Answer 16

• Computational creativity - goal of computational creativity is to model, simulate or replicate creativity using a computer, to achieve one of several ends
• To construct a program or computer capable of human-level creativity
• To better understand human creativity and to formulate an algorithmic perspective on creative behavior in humans
• To design programs that can enhance human creativity without necessarily being creative themselves

Question 17

divergent thinking

Answer 17

discovering several possible solutions
• Divergent thinking task used for assessment of domain-general creative cognition
 involves generating several possible solutions to an open-ended problem

Question 18

Brain network interaction during domain-general creative cognition

Answer 18

• Highly creative participants showed increased coupling of default network regions with the left inferior frontal gyrus, a region associated with cognitive control that is widely implicated in studies of divergent thinking
• Creative thought may benefit from cooperation of default + control network regions

Question 19

Brain networks and creative cognition: an integrative framework

Answer 19

• Creative cognition involves dynamic interaction of default + control networks
• Default network influences generation of candidate ideas, but control network can constrain + direct this process to meet task-specific goals via top-down monitoring + executive control
• Important role of Default network in episodic memory retrieval
 Memory systems might have key role in generation of candidate ideas
•

• the default network may provide bottom-up evaluations via spontaneously generated and self-referential mechanisms; the control network, in turn, may compare this information to the task goal and modify it via cognitive control mechanisms such as inhibition and selection

Question 20

Article: Generating original ideas: The neural underpinning of originality (Mayseless

Answer 20

• Create and original indviduals have a more frlexible associative networke as they are able to abandon more conventional thinking paths to connect different stimuli in new and diverse ways
• indicate that the ability to produce original ideas is mediated by activity in several regions that are part of the DMN including the medial prefrontal cortex (mPFC) and the posterior cingulate cortex (PCC)

 According to the dual model of creativity, the creative process involves a cyclic movement between the process of associative generation of ideas and the process of evaluating these ideas against a benchmark of standards
 creativity may reside in the balance between an associative network allowing free flow of ideas (default mode network) and a network supporting the inhibition of irrelevant ideas (executive control network)

Question 21

hill climbing

Answer 21

A typical process starts with an existing design, perhaps an earlier one that needs to be improved or extended, or a design for a related task. The designer then makes changes to this solution and evaluates them. S/he keeps those changes that work well and discards those that do not and iterates. It terminates when a desired level of performance is met, or when no better solutions can be found—at which point the process may be started again from a different initial solution
 Much of the space remains unexplored and many good solutions may be missed
 Current machine learning methods also based on it
 Neural networks and deep learning follow a gradient that is computed based on known examples of desired behavior  the gradient species how the neural network should be adjusted to make it perform slightly better, but it also does not have a global view of the landscape, i.e. where to start and which hill to climb
 reinforcement learning starts with an individual solution and then explores modifications around that solution, in order to estimate the gradient
 With large enough networks and datasets and computing power, these methods have achieved remarkable successes in recent years

Question 22

• Creativity Support Tools (CST

Answer 22

tools used by computers that influence, support and enhance creativity

Experts: require tools that enable them to rapidly iterate through + document many different ideas early in creative process

Novices: require low threshold of entry into creative domain because they do not have confidence, skill or motivation to withstand engagement with tool (Garage Band for people who don’t play)

Question 23

Creativity (Boden)

Answer 23

• Creativity – ability to generate novel, and valuable ideas
 Valuable – Interesting, useful, simple, richly complex etc.
 Ideas – Concepts, theories, interpretations, sculptures, graphic images etc.
 Novel – Novelty can have two importantly different meanings

Question 24

Difference Historical & psychological novelty

Answer 24

 Psychological-Creativity – Novelty that is new to the person who generated it
 Historical-Creativity – Novelty that is P-creative and has never occurred before

Question 25

three different ways in which creativity happens:

Answer 25

combination, exploration, transformation

Question 26

combination

Answer 26

Produces unfamiliar combinations of familiar ideas and works by making associations between ideas that were previously only indirectly linked most difficult for computers• Certainly, AI programs can make fruitful new combinations within a tightly constrained context, but no current AI has access to the rich and subtly structured stock of concepts that any normal adult human being has built up over a lifetime. • One of the best current computer models of combinational creativity is the joke generating system JAPE that can compose nine different kinds of jokes.

Question 27

 Exploration

Answer 27

Rests on some culturally accepted style of thinking or Conceptual Space which is defined by a generative (implicit) rules. The person moves through the space, exploring it to find the limits and potentials of the space in question. * Exploratory creativity can also be modelled by AI, provided that the rules of the thinking style can be specified clearly enough to be put into a computer program. * Despite the difficulties, there has been much greater success than in modelling combinational creativity and in many exploratory models the computer comes up with results that are comparable to those of highly competent human professionals. * For example, AARON is a computer program that creates original artistic images.

Question 28

Transformation

Answer 28

The space or style itself is transformed by altering one or more of its defining dimensions. As a result, ideas can now be generated that simply could not have been generated before the change. * Many people believe that a computer could never achieve transformational creativity, because after all, a computer does what its program tells it to do, and no more. * However, what this reasoning ignores is that the program may include rules for changing itself, called Genetic Algorithms (GAs) * GAs can make random changes in the program’s own task-oriented rules and these changes are similar to the point mutations and crossovers in biological evolution. * Many evolutionary programs also include a Fitness Function, which selects the best members of each new generation of task programs for use as “parents” in the next round of random rule changing.

Question 29

major challenges in creating creative programs

Answer 29

 Accumulation – By accumulating knowledge of what is known and building models of the world, a program is better equipped to solve new problems.  Reflection – Reflection is thinking at the metalevel and included knowledge about tasks and problem-solving procedures. To introduce reflection, we could shift representations, introduce new ways to satisfy constraints, introduce randomness or reflect on and change values.  Transfer – Representation is the key to improving the capabilities of programs by sharing ontologies, using analogy engines, importing concepts from an old domain and modifying previously successful methods.

Question 30

Creativity (Dahlstedt)

Answer 30

* Computational creativity – explore potential of our machines to be creative in their own right & to act as autonomous creators/co-creators * Exploratory creativity – explore space as it is defined by the problem, looking for previously undiscovered states of unexpectedly high value * Transformational creativity – actively transform the problem space, redefining criteria that shape the space (exception rather than the rule, e.g. development of music)

Question 31

How can one search in such a huge or infinite space in a more structured way? (optimization techniques)

Answer 31

• Hill-climbing: sub-regions can be found by random sampling (once an interesting point is found we can look at nearby points and evaluate them and choose the path with the best option and go that direction and so on)  very bad visibility: not sure if when at peak this is actually peak • Principle of Darwinian evolution: start with a set of (random) points, evaluate them, and based on which ones are best or most interesting you create a new set of points (“offspring” points that inherit something from their “parent” points)  this way the space is searched for better and better points (according to current evaluation criteria) in a structured search process that balances chance and control)  climbing all hills at the same time  way more efficient

Question 32

sow, cultivate , harvest

Answer 32

• Practice changes from a kind of design and construction of artefacts to the harvesting of results form infinite result spaces, reaping consequences from chaotic or unpredictable complex principles in art (through the use of generative search techniques) • You can now sow, cultivate and harvest • Sow: you construct or define the space which equals filling it with content but without exact control of details – and this is crucial (if you know all the details it is too small to be interesting) • Cultivate: explore the space in search for interesting results and thus realizing/generating the potential in the visited parts of the space • Harvest: the best ones are brought back  the experience of exploring such spaces (sowing, cultivating and harvesting of strongly unpredictable results) is strong

Question 33

Adaptive novelty

Answer 33

– producing something that is both novel and adaptive in its context (i.e., cold weather adaptation, inventions, …) • Arriving at adaptive novelty is a process of searching through a space of possibilities, also called the problem space (by Newell and Simon). In this space, a step and the solution are well-defined

Question 34

A system has creative autonomy if it meets the following three criteria:

Answer 34

1) Autonomous evaluation – system can evaluate its liking of a creation without seeking opinions from an outside source 2) Autonomous change – the system initiates, and guides changes to its standards without explicitly directed when and how to dos so 3) Non-randomness – the system’s evaluations and standard changes aren’t purely random  Creative autonomy may be necessary to but not sufficient (gives the potential to be creative but does not mean that AI will actually be creative)

Question 35

supervised learning

Answer 35

1. Supervised learning (SL) policy network – training takes place directly from expert human moves; provides efficient learning updates with immediate feedback  Input s to the policy network is a simple representation of the board state  Trained on randomly sampled state-action pairs (s, a)  13-layer network – larger networks achieve better accuracy but are slower to evaluate during search

Question 36

Reinforcement learning

Answer 36

2. Reinforcement learning (RL) policy network – improves SL policy network by optimizing the final outcome of games of self-play; adjusts the policy towards the correct goal of winning games rather than maximizing predictive accuracy  Identical in structure to SL policy network  Play games between current policy network and a randomly selected previous iteration of the policy network  Won more than 80% of games against SL policy network; won 85% of games against Pachi (sophisticated Monte Carlo search program)

Question 37

Value network

Answer 37

3. Value network – predicts the winner of games played by RL policy network against itself; reinforcement learning of value network  Involves a similar architecture as the policy network, but outputs a single prediction instead of a probability distribution

Question 38

The neural networks are trained through several stages of machine learning:

Answer 38

supervised learning, reinforcement learning, value network

Question 39

combinational creativity

Answer 39

unfamiliar combinations out of familiar ideas JAPE: joke robot Most difficult AI (lack of cultural knowledge and intuition)

Question 40

transformational creativity

Answer 40

space is changed by different rules: a whole new way of thinking Rules for changing itself; e.g. genetic algorithms with random alterations (minor mutations (Sims‘s program))

Question 41

Generative Art

Answer 41

human artists write a program, and then stand back and let it run

Question 42

Evolutionary Art

Answer 42

computer produces novel results, unpredicted by the artist

Question 43

Interactive Art

Answer 43

computers performance is affected by outside events, the movemênt or actions of humans

Question 44

Creative ideas: combining new and old, applying ideas from one discipline to the other

Answer 44

Preparation: work very hard but with no solution Incubation: set the problem aside Illumination: solution suddenly appears Elaboration: inspection of that solution

Question 45

Blind Variation:

Answer 45

Campbell argued that there is nothing special about creativity at all & purely by chance, some random event could suggest a solution to a problem

Question 46

Neural-network theory

Answer 46

If these nodes are already strongly connected, thinking is routine & unsurprising If there is a weak or undirected connection, the connection can in principle be strengthened, which provides a mechanism for Selective Retention (This process could also be called creative insight or classical conditioning)

Question 47

Defocused attention (Mendelsohn)

Answer 47

Less focused attention leads to more creativity More ideas attended = more possible creative combinations During incubation: nodes representing the problem are set aside but still partially activated Other nodes active: overlap through a weak connection to problem nodes = insight

Question 48

Less focused attention leads to more creativity More ideas attended = more possible creative combinations During incubation: nodes representing the problem are set aside but still partially activated Other nodes active: overlap through a weak connection to problem nodes = insight

Answer 48

Focused Attention: few nodes activated; strong lateral inhibition on other nodes Steep activation = focused, not creative Defocused Attention: larger number of nodes moderatly activated (diffuse); less lateral inhibition Flat activations = less focused, more creative

Question 49

Primary/ secondary processes of thinking (Kris)

Answer 49

Primary: free associative, analogical (defocused) Secondary: logical, goal oriented (important for the elaboration part) Alternation between those increases creativity

Question 50

Creativity and Arousal (Martindale)

Answer 50

Each node receives direct input from other nodes and from the arousal system Decrease in arousal: more nodes will be activates and attention is defocused = variable behavior Increase in arousal: one/ two nodes have all activation; attention is focused = stereotyped behavior Everything increasing arousal decreases flow in creativity

Question 51

Simulated Annealing (Hopfield)

Answer 51

Activation of a node depends on temperature; nodes are attracted to a minimal state of energy „Probalistic technique to approximate a global optimum“ High Temperature: nodes behave more random, are activated no matter what (defocused attention) Low Temperature: nodes activated only if there is proper input (high attention)

Question 52

blind variation

Answer 52

creativity occurs purely by chance

Question 53

defocused attention, associative hierarchies, primary process thinking

Answer 53

all propose that creativity is linked to less focused attention , primary process thinking, little lateral inhibition, low arousal

Question 54

global minimun

Answer 54

solution with least effort

Question 55

selective retention

Answer 55

creativity can arise when there is a weak connection between two concepts => strengthening leads to creative insight

Question 56

• investment theory of creativity

Answer 56

Sternberg & Lubart) – a creative computer must be able to articulate its sense of how a particular product or idea can be “bought low and sold high”

Question 57

big c

Answer 57

ig-C creativity & little-c creativity 1) Big-C must not be a function of big groups  Big-C researchers explore the collective generation and acceptance of new ideas  particularly relevant for organizational behavior, but should also be applied to smaller microsystems and subgroups like classrooms, schools, students, and teachers 2) Big-C creativity must not resolve extremely difficult problems

Question 58

artificial I in creativity

Answer 58

• Deep learning programs operate with interconnected neural networks that make sense of imputed data (like humans) • Through the method of trial-and-error deep learning programs make “intelligent guesses” that become increasingly precise with new iterations of probability analyses • AI might soon replace humans even in professions that require complex cognitive and social-emotional skills  As AI develops further • Advantage: capacities to make important decisions without emotional attachment to intended beneficiaries  but still AI algorithms reflect the interests of programmers, so questions of programmer bias arise (e.g., face recognition software more accurate at identifying the race and gender of white males than non-white females)  success of tasks done by machines depends on the ways programmers are instructing them to find and solve problems • Sophisticated machine learning programs are dependent on input from human minds  AI is fed information from an existing knowledge base  the artificial domain is closed as it is created from the limitations of the human mind and any further iteration is dependent on the inputs of human creators • AI engines are (as of now) incapable of radically altering existing paradigms independently  while Deep Dream indeed presents admirable artworks they pale in comparison to the lasting innovations of Da Vinci or Picasso

Question 59

divergent thinking

Answer 59

ability to produce many alternate ideas

Question 60

boden

Answer 60

 They do not accumulate experience and, thus, cannot reason about it.  They work within fixed frameworks, including fixed assumptions, methods and criteria.  They lack the means to transfer concepts from one program to another

Question 61

• A system has creative autonomy if it meets the following three criteria:

Answer 61

1) Autonomous evaluation – system can evaluate its liking of a creation without seeking opinions from an outside source 2) Autonomous change – the system initiates, and guides changes to its standards without explicitly directed when and how to dos so 3) Non-randomness – the system’s evaluations and standard changes aren’t purely random