These...

• include photos, screenshots, screengrabs, machinima, illustrations, cartoons, drawings, diagrams, artful visuals, and others
• tend to be decontextualized and disaggregated from the original use context
• may be stand-alone one-offs, or may be parts of sequences or sets or collections

image coding: labeling and categorizing the imagery for informational value

bottom-up coding by analyzing the captured images in the sets to identify insights

top-down coding by applying a theory, a research question, a hypothesis to the imagery to draw meaning

images may be analyzed in whole or in part

systematizing...for transferability, for rigor, for consistency, for repeatability/reproducibility

Some Types of Askable Research Questions from Social Image Sets

What are some conscious and subconscious associations with...

• A concept?
• A person?
• A phenomenon?
• A word?
• A cultural occurrence?
• A location?
• A technology?

How are different people groups and persons depicted in photos and visuals? In photos and visuals from different time periods? Geographies? Cultures?

How do memes instantiate visually?

Downloading Images En Masse after a Text-based Search

• text-based searches based on tagging (both folk-tagged by humans and machine-tagged by computers)
• can use wildcard characters (*) and boolean operators (+, -, and others)...to shape or narrow the sought image set
• may be scraped en masse using image-sharing platform-specific tools, like Flickr Downloadr (with a take in the hundreds)
• may be downloaded from the Web using browser add-ons, like Picture Downloader Professional on Google Chrome (expand each page as fully as possible, with a "take" about 1000 images or so depending on the image search in the web browser)
• may be captured using a scripted agent

Non-Text-Based Image Searches

• by filtering
• by descriptor (text-based)
• by similar imagery and machine vision (by similar-image training sets, by exemplars of one)

Image Set Curation

• removal of images that "clearly" do not belong to the set
• striving to preserve as many images to use as possible (least lossy)

Some Bottom-up Coding Methods

Image contents: Entities? Egos? Interrelationships? Locations? Objects? Time of day/ night / dawn / dusk? Indoors? Outdoors?

The scenario set-up: Serendipitously captured? Creator purpose? Amount of effort to set up?

Quality: How much pre-production? How much post-production? Color jumping? Faux effects?

Inspiration: Original inspiration? Memetic copying? Derivative? Aesthetic influences? Styles?

Fidelity: Photo-realistic or not? High or low fidelity? Non-fiction or fiction? Levels of polish?

Communications: Purposeful messaging? Audience?

Sophistication: Technologies used? Technologies not used?

Information sourcing? Original? Usurped? Mashups?

Single or Combinatorial: Singular image? Composite images? Layout?

Perspective: Point-of-view? Angle? Framing? Focal point?

Metadata: Location? Technologies? Parameter setting? Time of day? Photographer identity?

Approaches to Sensemaking from Social Image Sets

Human visual analysis is more powerful currently than commercially available computation-based analytics in terms of nuance and interpretation...but it does not scale all that well.

General Research Approaches

The research approaches are generally the following:

• Categorization of images by types (creating classifications)
• Frequency counts?
• Themes?
• Recurring visual elements?
• Symbolism?
• Visual styles?
• Messaging?
• Subliminal insights? Latent messaging? (Manipulated messaging?)
• Mainline contents? Outliers?
• Utilitarian applications? Decision-making? Awareness? Design applications?

And?... So?...

Walk-throughs and Demos

• Flickr Downloadr
• Picture Downloader Professional (on Google Chrome Web Browser)

Why Use Social Imagery for Research?

Social imagery is plentiful. It is openly available. The cost to scrape imagery is low. This approach is not yet broadly exploited.

Imagery is multi-faceted and complex.

There are cultural practices and technologies that affect how visuals are created.

There are intended and unintended messages (and information). There are direct intended messages but almost always leaked data available as well. People have unintended "tells" because of how the subconscious and unconscious work. It is hard to suppress true internal states and hard to mask...even as people may enjoy their pretend versions of the world.

Manipulations by single entities may be more difficult at the higher level of social imagery collection around particular phenomenon...unless the contributor of the visuals is dealing with a rare topic at a particular slice-in-time.

Some Initial Lessons Learned

Use a range of search terms to "seed" image searches.

Social image sets are time-sensitive, especially with more dynamic topics.

Different social media search spaces result in quite different results.

Download social imagery at the highest resolution possible, not thumbnails.

Preserve original image names where possible. Preserve image metadata where possible.

Some images may deserve a reverse image search for further exploration.

It helps to code iteratively. Human attention is limited.

Assertability is limited from the data and analysis...as it is in virtually all research contexts. It helps to qualify any assertions with the limits of the image collection and the analysis.

It helps to use machine vision for some coding as well, but these are for mainly two things currently: objective identification (at scale) and sentiment analysis (at scale).

It helps to explore an image set in depth.

A researcher has to be comfortable with plenty of ambiguity and to hold interpretations provisionally.

Presenter Info

Dr. Shalin Hai-Jew

• Kansas State University
• shalin@k-state.edu
• 785-532-5262