Part One

How does one built-in algorithm such as Inceptionv3 in ODM cluster the data?

Inception V3, a deep convolutional neural network, is utilized within Orange Data Mining to analyze and cluster image data. This algorithm excels at detecting intricate features in images through layers that learn hierarchical representations. In this specific application, it appears that the algorithm clusters images not only by subtle color gradations but also by visual patterns and compositional elements. For instance, paintings with predominant pink hues are aggregated on the left-hand side of the grid, suggesting a color-based clustering. Similarly, artworks featuring geometric patterns, such as rectangles and circles, are grouped towards the bottom of the grid. This indicates that Inception V3 effectively discerns and clusters based on both color palette and the presence of distinct shapes and structures within the paintings

Do other algorithms give you different results?

Painters: Paintings by the same artist are grouped closely together. This indicates that the painters embedder is effectively capturing stylistic signatures that are unique to each artist. For example, most of Sage’s works are at the top of the frame instead of the bottom now.

What features seem to be most characteristic of the different quadrants of the image plot?

Top: Characteristics: This quadrant includes a mix of seemingly modern and classical elements, with a focus on structured, often stark compositions. The images, such as those by Sage (Sage11, Sage3), show a penchant for subdued palettes and architectural elements. Artistic Styles: Sage’s works here, featuring architectural and sculptural elements, are paired with images that maintain a similar structural integrity, suggesting that the embedder values compositional rigor and perhaps thematic gravity.

Right: Characteristics: Dominated by more vividly colored and dynamically composed images. This quadrant blends abstract and surreal elements with a clear presence of human figures and narrative hints. Artistic Styles: The pairing of Man Ray’s explorative, surreal images with Picasso’s vibrant cubist works (e.g., Picasso near the edge) and Sage’s surreal landscapes and scenes (Sage4, Sage2, Sage8) suggests a clustering based on narrative depth mixed with experimental visuals.

Left: Characteristics: This quadrant groups darker, moodier images that often utilize a limited color palette. The focus seems to be on abstract and surreal expression with an emphasis on emotional or evocative content. Artistic Styles: The clustering of Toyen’s deeply surreal artworks (Toyen1, Toyen2, Toyen5) suggests an affinity for darker themes and more introspective or dream-like qualities.

Bottom: Characteristics: Features a mixture of abstract forms and more clear, representational elements. This quadrant is characterized by a balance of color and form, where abstract elements are integrated with recognizable subjects. Artistic Styles: Here, Miró’s vibrant abstracts (Miró2, Miró9, Miró15) are mixed with Picasso’s figurative and abstract pieces, indicating a clustering based on a playful yet thoughtful exploration of form and color.

Using hierarchical clustering, isolate specific clusters to look more closely.

1. Common Features Across Both Artists’ Works: Abstract Forms: Both artists often employ abstract forms that distill figures and scenes to their essential elements. This simplification can involve both geometric and organic shapes. Color Use: Both Picasso and Miró are known for their vibrant use of color, but they also integrate muted tones effectively within their compositions. The clustering may have identified a similar approach to balancing color palettes. Surreal Influence: Miró’s works are inherently surreal, and while Picasso is primarily known for Cubism, he also explored surrealism, especially in his abstract figurative works.

2. Possible Reasons for Grouping Together: Thematic Similarities: Themes such as the human condition, dreams, and mythology may appear in both artists’ works, providing a deeper level of similarity that the clustering algorithm could pick up on. Innovative Techniques: Both artists experimented with form and perspective, which could make their artworks appear similar through the lens of a clustering algorithm focused on these aspects.

3. Interpretation of Hierarchical Clustering Results: Shared Aesthetic and Conceptual Approaches: The clustering suggests that both artists share a common ground in how they abstract reality to express deeper meanings. Their works might also parallel in how they manipulate space and color to create a sense of depth or flatness, which could be a significant factor in their grouping. Sensitivity of the Clustering Algorithm: This outcome highlights the algorithm’s sensitivity to not just visual similarities but also to the artists’ shared approaches to abstraction and surrealism. This can be particularly insightful in studies where cross-artist themes and influences are of interest.

1. All three paintings prominently feature geometric shapes and structures. Man Ray’s painting integrates these elements in a more surreal, organic context, while Sage’s paintings focus on structured, almost architectural settings and surreal landscapes. The use of lines and structured forms is quite evident, creating a sense of space and depth that plays with the viewer’s perception.

2. Surreal and Abstract Themes: Each artwork incorporates a surreal quality that distorts reality in a subtle yet impactful way. This surrealism isn’t flamboyant but rather integrated into the structure of everyday objects and spaces, making the ordinary appear unfamiliar. The abstract qualities in these works suggest a departure from realistic depiction, favoring instead an exploration of form and the interplay of reality with imaginative constructs.

3. Muted and Harmonious Color Palettes: The color schemes in all three artworks are subdued and harmonious, relying on a limited palette to enhance the compositions without overwhelming the geometric and surreal themes. This use of color supports the structural elements rather than taking a forefront position, which ties the pieces together in a visual and thematic narrative.

Part two

How well did any of the built-in algorithms you chose predict the categories you established?

• High Accuracy for Some Artists: The classifier performs very well for Miró, Picasso, and Sage, with high true positive rates, indicating that it can effectively recognize and differentiate their styles.
• Common Misclassifications: ManRay and Toyen: There are confusions between ManRay and other artists like Picasso and Sage, and Toyen is sometimes confused with ManRay. This could suggest some stylistic similarities in certain works or a challenge for the model in distinguishing unique features of these artists. Picasso Misclassifications: Picasso is sometimes misclassified as ManRay, which could indicate overlapping features or styles in certain paintings that the model finds confusing.
• The performance of Inception V3 is mixed compared to the painting embedder. It shows a drop in accuracy for Picasso and Sage, suggesting that while it might handle some artists better, it does not universally improve classification across all artists.
• Misclassification Patterns: Man Ray and Picasso: A notable amount of Man Ray’s work is misclassified as Picasso’s, which could indicate overlapping features in their styles or abstraction techniques that confuse the model. Sage and Toyen: The minimal misclassifications between Sage and Toyen suggest that while they are generally well-classified, there may be occasional overlaps in style or thematic elements that are not as distinct to the model.
• Overall Model Performance: The model performs very well for artists like Sage and Miró, who have relatively high accuracy, but struggles more with Man Ray. This variation in performance might be due to the inherent differences in the artists’ styles or the characteristics of the artworks sampled for each artist.

1. Consistency and Accuracy in Color Detection:
   • Greys: Images labeled as grey generally show a dominant presence of grey or subdued tones. This indicates that the algorithm effectively identifies and categorizes images where grey tones are prevalent, which often correspond to images with metallic, urban, or shadowed elements.
   • Blues and Reds: The categorization of blues and reds shows a strong consistency, with images prominently featuring these colors correctly labeled. This suggests good model accuracy in recognizing vibrant and distinct colors which are easier to distinguish.
   • Yellows: Yellow-labeled images seem to capture the essence of the color well, highlighting artworks where yellow plays a significant role either in brightness or as a focal element.
2. Challenges and Ambiguities:
   • Pink and Black: These categories seem less populated and may represent challenges in the model’s ability to distinctly recognize these colors when they are less dominant or mixed with other hues.
   • Other: This category is quite diverse, suggesting it includes images where no single color dominates or the dominant color does not fit neatly into the other predefined categories. It seems to serve as a catch-all for complex images, which could be problematic if looking for more granular insights.

• High Accuracy for Grey: Grey has the highest number of correct predictions, indicating that the model is effective at recognizing and classifying grey tones.
• Significant Confusion with ‘Other’: The ‘Other’ category has a high degree of misclassification, both from and to other categories. This suggests that ‘Other’ is either too broad or that its defining characteristics are not well understood by the model.
• Challenges with Less Represented Colors: Black, blue, pink, red, and yellow show relatively high rates of misclassification. This might be due to fewer training samples or the model’s difficulty in distinguishing these colors within the context of artwork.

If you could train your own algorithm what would you aim to teach it?

• Enhancing the model’s ability to distinguish between closely related or poorly represented colors could improve accuracy. This might involve training with a more balanced dataset where each color is equally represented.

• Refining the color categories or perhaps defining them in a more nuanced way could also help, especially for categories like ‘Other’, which appears too ambiguous.

Ready for grading

I chose to analyze Taylor Swift’s song lyrics. I divided her works into three eras, 2006 to 2010, 2011 to 2018 and 2019 to 2023. Each era includes three to four albums, and they all have different attributes.

-2006 to 2010

• This era covers Taylor Swift’s early career, characterized by country-pop and country-rock influences.
• Focus on themes such as young love, heartbreak, and coming-of-age experiences.
• Key albums include “Taylor Swift” (2006), “Fearless” (2008), and “Speak Now” (2010).

Most frequently used phrases are “i’m” and “you’re”, which also appear a lot in the next two eras. These phrases unfold that Taylor tells the stories through first person perspective and is very good at portraying her relationship with others. However, ‘you’re’ doesn’t solely refer to others but can also encompass Taylor herself. For instance, in songs like ‘Fifteen,’ she adopts the perspective of both her younger self and her audience, crafting lyrics that read like a heartfelt letter

The frequent use of words like ‘like’, ‘love’ and ‘baby’ suggests a predominant theme of romantic relationships in her songs during this era. According to the trends chart, ‘like’ and ‘love’ almost appear all the time.

Additionally, the frequently used word “come” may imply themes of anticipation, invitation, or the desire for closeness, further reflecting the romantic narrative prevalent in her music during those years. The links tool indicates that ‘come’ is frequently paired with ‘on’ and ‘back’. The combination of “come on” could convey encouragement, urging someone to join or participate, while “come back” might evoke sentiments of nostalgia, longing, or the hope for a return to a previous state or relationship.

-2011 to 2018

• This period marks Taylor Swift’s transition to mainstream pop music, incorporating elements of synth-pop and electronic dance music (EDM).
• Themes may include empowerment, self-discovery, and personal growth.
• Key albums include “Red” (2012), “1989” (2014), and “Reputation” (2017).

Obviously, ‘i’m’ is more frequently used than ‘you’re’ in this era. In “Red,” Taylor explores themes of love, heartbreak, and self-discovery, with tracks like “All Too Well” and “I Almost Do” showcasing her emotional vulnerability. Similarly, “1989” looks into her experiences with love and relationships, with songs like “Style” and “Wildest Dreams” reflecting her own perspectives and feelings. “Reputation” sees her confronting public scrutiny and personal challenges, with tracks like “Look What You Made Me Do” and “Delicate” expressing her emotions and experiences in a more assertive and introspective manner. Therefore, the increased frequency of the phrase “i’m” compared to “you’re” during this period may indeed suggest a focus on her own feelings and experiences in her music.

‘Like’ and ‘love’ are still very consistently used words in this era. It’s noticeable that the wordtree indicates the words frequently paired with ‘love’ are ‘hurts’, ‘bleeds’ and ‘fades’. It suggests a nuanced exploration of the complexities of love in Taylor Swift’s song lyrics. These words evoke themes of pain, vulnerability, and the transient nature of relationships. It reflects a deeper narrative where love is portrayed not only as a source of joy but also as something that can cause anguish and uncertainty. This use of language adds layers of depth to her storytelling, capturing the emotional rollercoaster often associated with love and relationships.

Different from her earlier era, where words like ‘come’ and ‘love’ dominated, the period spanning albums “1989” and “reputation” witnesses a thematic shift in Taylor Swift’s lyrical content. In this phase, phrases such as ‘shake’, ‘dancing’, and ‘run’ are notably recurrent, indicating a deliberate departure from her previous themes. This change suggests Taylor’s response to the rumors and scrutiny surrounding her, with a focus on themes of liberation, defiance, and self-empowerment. The repeated use of words associated with movement and freedom reflects her determination to break free from constraints and assert her independence amidst adversity.

-2019 to 2023

• This era reflects Taylor Swift’s experimentation with diverse musical styles and more mature lyrical content.
• Themes may include introspection, social commentary, and embracing authenticity.
• Key albums include “Lover” (2019), “Folklore” (2020), “Evermore” (2020), and “Midnights”(2022).

The increased frequency of “I’m” compared to “You’re” suggests a shift in focus towards Taylor Swift’s personal experiences and perspective during the 2019 to 2023 era. This emphasis on the first-person narrative indicates a deeper exploration of self-expression and introspection in her songwriting. Additionally, the significant occurrence of “I’d” further reinforces this introspective theme, as it implies contemplation and reflection on past events or hypothetical scenarios. Overall, these trends suggest a more introspective and self-aware approach to storytelling in Taylor Swift’s music during this era, characterized by a heightened sense of personal reflection and vulnerability.

The consistent appearance of “night” throughout the lyrics in all three eras, often paired with “middle” and “dream,” hints at a persistent motif of introspection and the passage of time. The combination of “night” with “middle” suggests moments of transition or ambiguity, while its connection with “dream” implies contemplative or imaginative states. This repetition underscores the symbolic importance of “night” in Swift’s songwriting, contributing to themes of self-reflection, aspiration, and the temporal journey depicted in her music.

The presence of both “daylight” and “night” in the lyrics of Taylor’s songs during the 2019 to 2023 era suggests a thematic shift towards a more nuanced exploration of time and emotional states. Unlike the absence of “daylight” in the earlier eras, its inclusion alongside “night” implies a broader spectrum of experiences and emotions being depicted in her music during this period. This contrast between light and dark imagery could symbolize a deeper introspection into the complexities of life, with “daylight” representing hope, clarity, and optimism, while “night” conveys darkness, uncertainty, and introspection. Overall, the presence of both “daylight” and “night” in the lyrics of this era suggests a more multifaceted and mature exploration of emotions and experiences in her songwriting.

The inclusion of “golden” alongside “daylight” suggests a sense of warmth, happiness, and positivity associated with moments of clarity and optimism. This pairing evokes imagery of bright, sunlit days filled with joy and contentment, contrasting with the darkness and uncertainty often associated with “night.” Together, “daylight” and “golden” create a vivid picture of moments of happiness and fulfillment, further enhancing the multifaceted exploration of emotions and experiences in Taylor Swift’s songwriting during this period.

Throughout Taylor Swift’s career, her lyrics reflect a profound evolution and maturation. In the early years, frequent mentions of “you’re,” “love,” and “like” paint a picture of a young girl navigating the complexities of love and heartbreak. As she transitions into the second era, characterized by hits like “Shake It Off,” Swift begins to assert herself, shaking off rumors and embracing her individuality. This phase sees her exploring themes beyond romance, delving into self-discovery and empowerment. By the third era, Swift emerges as a seasoned storyteller, her lyrics demonstrating a newfound maturity and depth. Addressing themes of friendship and feminism, she showcases a more nuanced understanding of the world and her place within it, marking a significant evolution in her artistry.

Conclusion

Before I began the analysis, I had a basic understanding of the context of Taylor Swift’s albums and her personal evolution. Computer-assisted analysis allows for the identification of patterns, trends, and correlations within a large corpus of text that may not be immediately apparent through a linear read. It clearly visualizes the changes through different times. But when I use the DreamScape tool, it doesn’t always accurately show the place, and I don’t know how to change that information, otherwise it would be fun to analyze the location mentioned in Taylor’s song. The format of the corpus, provided in textual form, was suitable for the purposes of computer-assisted analysis. However, some shortcomings of the texts include variations in lyric transcriptions, potential inaccuracies in the representation of song lyrics.

To evaluate the efficacy of Voyant tools with Chinese text, I selected songs from Taylor Swift’s third era and inputted both the English and corresponding Chinese lyrics into the tool. While the word cloud displayed minimal variation between the languages, discrepancies arose in translations, notably with some English phrases fragmented into individual Chinese characters. For instance, “Cornelia Street” was divided phonetically into five distinct characters, resulting in their isolated appearance in the Chinese word cloud. However, the links tool yielded striking disparities, the English analysis revealed numerous correlated word pairs, while the Chinese counterpart generated only a single pair. I speculate it’s because Voyant tools understands and works better with English words.

Reference:
Swift, Taylor. Taylor Swift, Apollo A-1 LLC, 2006.
Swift, Taylor. Fearless (Taylor’s Version), Taylor Swift, 2021.
Swift, Taylor. Speak Now (Taylor’s Version), Taylor Swift, 2023.
Swift, Taylor. Red (Taylor’s Version), Taylor Swift, 2021.
Swift, Taylor. 1989 (Taylor’s Version), Taylor Swift, 2023.
Swift, Taylor. reputation, Apollo A-1 LLC, 2017.
Swift, Taylor. Lover, Taylor Swift, 2019.
Swift, Taylor. folklore, Taylor Swift, 2020.
Swift, Taylor. evermore, Taylor Swift, 2020.
Swift, Taylor. Midnights, Taylor Swift, 2022.

Ready for grading.

The the data from the file ‘All_culture_information.csv’ reveals significant disparities in the distribution of objects across different cultural categories. Out of the 254 rows, it is evident that American, German, French, Italian, British, and Japanese objects dominate the collection, with each category containing more than ten thousand objects. Byzantine, Greek, Chinese, Roman, and Dutch artifacts are also well-represented, with each category having more than five thousand objects. Conversely, certain cultures, such as Lebanese, Baule, Central African, Turkistan, and Yemeni, are notably underrepresented, with only one piece each in the collection.

Historical Significance: Countries like America, Germany, France, Italy, Britain, Japan, Byzantine, Greece, China, Rome, and the Netherlands have rich cultural and artistic histories that have produced a significant number of artworks over the centuries. These countries have been centers of artistic production and patronage, leading to a large number of artworks being created and preserved.

Colonialism and Imperialism: The dominance of certain countries in the museum’s collection may reflect historical processes such as colonialism and imperialism. Countries like Britain, France, and Italy had extensive colonial empires, which allowed them to acquire artworks from the regions they colonized. This resulted in a disproportionate number of objects from these countries compared to others.

Cultural Exchange and Trade: Countries that were major centers of trade and cultural exchange, such as Italy during the Renaissance and Japan during the Edo period, likely produced a large number of artworks that were traded internationally. This would explain the abundance of Italian and Japanese objects in the collection.

Preservation Efforts: Some countries may have had more robust efforts to preserve their cultural heritage, leading to a larger number of artworks being available for acquisition by institutions like the Harvard Art Museum. Countries like America, Germany, France, and Britain have well-established museum systems and cultural institutions that actively collect and preserve artworks.

Accessibility and Availability: The limited representation of certain countries like Lebanon, Baule, Central African, Turkistan, and Yemeni could be attributed to factors such as accessibility and availability. Artworks from these regions may be less accessible due to political instability, limited resources for preservation, or challenges in acquiring them through legal channels.

Some of the Most Viewed Object

In terms of the most and least viewed objects, I searched Chinese and Czech in the noebook Harvard_API_All_objects.

The most viewed Chinese artifact is One of a Pair of Dragonhead Bangle Bracelets. This piece of jewelry, which has been viewed 527 times, was created between the 1850s and 1870s, and it was accessioned by the Harvard Art Museum in 1895

Jewelry in Chinese culture often carries deep symbolic meanings, reflecting themes such as wealth, status, and auspiciousness. Dragon motifs, commonly found in Chinese art and artifacts, symbolize power, strength, and good fortune. The dragonhead design of the bracelets likely signifies these auspicious qualities, making them culturally significant adornments.

The mid-19th century was a tumultuous period in Chinese history, characterized by internal rebellions, foreign invasions, and the decline of the Qing dynasty. The acquisition of the bracelets in 1895, amidst these historical events, may have symbolic significance, representing a connection to China’s rich cultural heritage amidst periods of change and upheaval.

The most viewed Czech artifact is Barrandov Terrace Restaurant. It’s a photograph taken by Jan Lauschmann in 1931.

The time period in which the photograph was taken, 1931, was a significant era in Czech history. This was during the interwar period, characterized by social, political, and cultural changes in Czechoslovakia. The image of the Barrandov Terrace Restaurant captures a snapshot of daily life and leisure activities during this dynamic period, making it historically intriguing to viewers interested in Czech history.

The Barrandov Terrace Restaurant was a prominent social and cultural hub in Prague during the early 20th century. It was frequented by artists, intellectuals, and members of high society, reflecting the vibrant cultural scene of the time. The photograph may hold cultural significance for Czech audiences nostalgic for the heyday of Prague’s café culture and social life.

Choice of Three Cultures

I chose American, Japnese and Chinese culture, as they are among the most prominent in terms of quantity in the museum’s collection.

The peak in the number of American objects accessed in 2011 (31.74k) could reflect concerted efforts by the Harvard Art Museum to expand its American art collection or commemorate significant milestones in American cultural history. This period may have witnessed increased acquisitions of American artworks through purchases, donations, or bequests, driven by factors such as institutional initiatives, fundraising campaigns, or the availability of high-quality artworks on the market.

The fluctuating number of Japanese artifacts accessed in recent years, ranging from 1 to 21, may suggest variability in collecting priorities, availability of artworks, or institutional resources within the Harvard Art Museum during this period. Factors such as changes in curatorial focus, donor contributions, or the emergence of new scholarship on Japanese art could influence the acquisition decisions.

Very interestingly, most of the Chinese objects(1278 pieces) were accessed in 1943. The year 1943 falls within a tumultuous period in Chinese history marked by the Second Sino-Japanese War (1937-1945) and World War II (1939-1945). China was facing significant political upheaval, territorial conflict, and social disruption due to Japanese invasion and occupation in various regions. Despite these challenges, cultural preservation and national identity remained important priorities for many Chinese scholars, artists, and cultural institutions.

The word cloud for American artifacts in the Harvard Art Museum shows that certain geographical locations like “United States,” “New York,” “York,” and “States” are prominent, indicating a focus on American regions or cities. The appearance of “Massachusetts” and “Ohio” suggests that specific states within the U.S. are also significant in the collection. The inclusion of “Lyonel Feinger” likely refers to an artist or individual associated with the artworks, potentially indicating a notable figure or a significant contributor to the American art scene. Overall, the word cloud highlights geographical locations, specific states, and possibly influential artists within the context of American artifacts in the museum’s collection.

The word cloud for Japanese objects indicates a focus on literary themes and cultural motifs. The prominence of words like “Series” and “Tale” suggests a significant presence of narrative or series-based artworks. “Genji,” “Fuji,” and “Mount Fuji” are prominent, possibly indicating a focus on iconic Japanese landmarks or literary references like “The Tale of Genji.” The repetition of “Fuji” and “Genji” underscores their importance. Additionally, terms like “Woman,” “Poems,” “Ichikawa,” “Actor,” “Courtesan,” and “volume” suggest a diversity of subjects, including representations of women, theatrical themes, and literary volumes. Overall, the word cloud reflects a rich tapestry of Japanese cultural and literary influences in the museum’s collection.

The word cloud for Chinese artifacts suggests a focus on form, decoration, and numerical symbolism. “Form” and “Decoration” being the largest words indicate an emphasis on the visual and ornamental aspects of the artifacts. The prominent numbers (“One,” “Three,” “Two,” “Four”) suggest numerical symbolism or possibly categorization within the collection. Additionally, the presence of terms like “birds,” “bamboo,” “flower,” “vase,” and “mountain” reflects the significance of nature and traditional Chinese motifs in the artworks. The appearance of “calligraphy,” “shuhua,” “Shizhuzhai,” and “Album leaf” indicates a strong presence of Chinese calligraphic and artistic traditions. Furthermore, the mention of specific items like “Circular Dish,” “Jade Disk,” and “Circular bowl” may highlight the Chinese aesthetic preference for circular shapes and forms. Overall, the word cloud reveals a diverse array of cultural and artistic themes within the Chinese artifact collection.

Ready for grading.

To add new posts, simply add a file in the _posts directory that follows the convention YYYY-MM-DD-name-of-post.ext and includes the necessary front matter. Take a look at the source for this post to get an idea about how it works.

Jekyll also offers powerful support for code snippets:

def print_hi(name)
  puts "Hi, #{name}"
end
print_hi('Tom')
#=> prints 'Hi, Tom' to STDOUT.

Check out the Jekyll docs for more info on how to get the most out of Jekyll. File all bugs/feature requests at Jekyll’s GitHub repo. If you have questions, you can ask them on Jekyll Talk.